In order for model ordering to work as expected in tests, we need to
keep two properties:
- Fixtures are all created in the past
- Models sort in the order that they were created
This allows us to do things like this:
post cards_path, params: { ... }
created_card = Card.last
When using UUIDv7 PKs rather than sequential integers, we have to make
sure a couple of things happen in order for this still to be true:
- Fixtures should generate deterministic IDs that translate to UUIDs
that would have been created in the past (i.e. before today)
- Newly created objects must have enough precision in their timestamps
so that they sort in the order they were created, and their random
component doesn't come into play.
To solve this, we use the deterministic numeric ID as a number of
milliseconds after an early year. And we ensure that the new timestamps
we create have sub-millisecond precision.
This matches the format in the migration files and makes the type_to_sql
override redundant.
The only wart we are left with is that since there are no true UUID
types in MySQL we'll have to assume that binary(16) columns are UUIDs.
In practice this is probably fine as the MySQL adapter doesn't map any
other types to binary.
We are not trying to handle user input so `cast` can be a no-op. Also
extract the base36 normalization logic, let the binary class handle the
serialize encoding and add some tests.
- Switch to binary 16 for UUID keys
- Remove AccountScopedRecord base class, all model use binary uuids now
- Fix the search sql to serialize uuids properly
- Patch the MySQL schema dumper to output binary lengths
These are mainly because the fixture's UUIDs are deterministic rather
than time-sortable, and more places where we need to correct the fixture
IDs as well.
Create search_index_0 to search_index_15 tables and shard each index by
account id. MySQL has no ability to pre-filter fulltext indexes by
another field so this is the best bet for improving performance.
Each fulltest index internally creates 11 sub tables (see
https://dev.mysql.com/doc/refman/8.4/en/innodb-fulltext-index.html) so
actually we have 192 tables in total here.
The search_index table name is generated dynamically based on the
account_id.
Add a single search_index table for full-text search of cards and
comments.
For the search there is a full-text index on the title and content
columns. The board_ids is also included in the table and accessible
board ids are pre-loaded and included in the search query. This allows
us to filter out inaccessible records before joining with other tables.
Right now the search is just using boolean search. This would give us
a bunch of syntax options
(see https://dev.mysql.com/doc/refman/8.4/en/fulltext-boolean.html)
except the search query filters those out.
I've removed the searchable_by method for now - everything is built
on the assumption that there's a single search index table and all data
must fit into it.
Queries are written in SQL, we don't have a SearchIndex ActiveRecord
model. That's because we'll likely want to shard the table and it will
be simpler to just keep with the raw SQL for that.
There's no stemming, highlighting or snippet extraction yet - we are
dumping the full description in the search results.
Data can be reindexed with the search:reindex rake task.
Kevin McConnell