Lean on ActiveRecord models for searching and strip out the raw SQL.
Replaces the search_index_* tables with sharded search_records_* tables
as that allows us to use a Search::Record model name.
A Class is dynamically created for each record table shard so that we
and we can access it via the Search::Record.for_account(account_id)
method.
Schema:
- add account_id to tables it was missing from
- make account_id a required column everywhere
- add [account_id] indexes, or add `account_id` to existing indices
Models:
- add `belongs_to :account` to all models (default to using a domain
model's account whenever possible)
- add account_id in all the necessary fixtures
- add account_id to insert_all hashes
- pass account_id to a few initialize calls
Miscellaneous:
- update the import script to set account_id
Note that I'm not adding account_id to the join tables primarily
because I couldn't think of an easy way to populate it without making
it a full Join model, and that was more work than I have time to take
on right now.
- Switch to binary 16 for UUID keys
- Remove AccountScopedRecord base class, all model use binary uuids now
- Fix the search sql to serialize uuids properly
- Patch the MySQL schema dumper to output binary lengths
These are mainly because the fixture's UUIDs are deterministic rather
than time-sortable, and more places where we need to correct the fixture
IDs as well.
Create search_index_0 to search_index_15 tables and shard each index by
account id. MySQL has no ability to pre-filter fulltext indexes by
another field so this is the best bet for improving performance.
Each fulltest index internally creates 11 sub tables (see
https://dev.mysql.com/doc/refman/8.4/en/innodb-fulltext-index.html) so
actually we have 192 tables in total here.
The search_index table name is generated dynamically based on the
account_id.
Add a single search_index table for full-text search of cards and
comments.
For the search there is a full-text index on the title and content
columns. The board_ids is also included in the table and accessible
board ids are pre-loaded and included in the search query. This allows
us to filter out inaccessible records before joining with other tables.
Right now the search is just using boolean search. This would give us
a bunch of syntax options
(see https://dev.mysql.com/doc/refman/8.4/en/fulltext-boolean.html)
except the search query filters those out.
I've removed the searchable_by method for now - everything is built
on the assumption that there's a single search index table and all data
must fit into it.
Queries are written in SQL, we don't have a SearchIndex ActiveRecord
model. That's because we'll likely want to shard the table and it will
be simpler to just keep with the raw SQL for that.
There's no stemming, highlighting or snippet extraction yet - we are
dumping the full description in the search results.
Data can be reindexed with the search:reindex rake task.
This wasn't working when creation and publishing happened on separated instances
of the same record, which is what happens when you edit a draft and then you
publish it
https://app.fizzy.do/5986089/cards/2835/edit
We do this from jobs, so there is no request id to discard the page refresh in the client. We
don't want page refreshes in these cases, as those are normally handled via the change that
originated this detection, in any case.
Removes the creating status completely as well as the abandoned cards system. It will
always resume drafts if they exist, for a given collection and user.
https://app.box-car.com/5986089/cards/2489
Donal McBreen