Only index cards and comments if their card is `published?`, using a
new method `searchable?` that is implemented on both Card and Comment.
This prevents draft cards and their comments from being indexed.
When drafts are published, the existing `update_in_search_index`
callback creates the record via `upsert!`, or else ensures unpublished
records are removed from the index.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
We're about to make a change to assert that draft cards are not added
to the search index, and so let's make the intention behind these
tests clear first.
To prevent overflowing the columns in the search index tables (and also
just to avoid creating massive indexes), let's limit the searchable
content to the first 32KB.
It doesn't actually work, and even if we could make it work reliably
we are better off if the records always know to go to the right shard.
It does make the interface a bit more complicated as we need to select
the right shard class with `for(account_id)`.
Instead we'll compute the table name dynamically based on
Current.account where needed. Also we'll prevent searchable records
from being saved if Current.account is not set, otherwise the after
commit callbacks will fail.
Lean on ActiveRecord models for searching and strip out the raw SQL.
Replaces the search_index_* tables with sharded search_records_* tables
as that allows us to use a Search::Record model name.
A Class is dynamically created for each record table shard so that we
and we can access it via the Search::Record.for_account(account_id)
method.
Create search_index_0 to search_index_15 tables and shard each index by
account id. MySQL has no ability to pre-filter fulltext indexes by
another field so this is the best bet for improving performance.
Each fulltest index internally creates 11 sub tables (see
https://dev.mysql.com/doc/refman/8.4/en/innodb-fulltext-index.html) so
actually we have 192 tables in total here.
The search_index table name is generated dynamically based on the
account_id.
Mike Dalessio