Shard the search index into 16 tables

Create search_index_0 to search_index_15 tables and shard each index by
account id. MySQL has no ability to pre-filter fulltext indexes by
another field so this is the best bet for improving performance.

Each fulltest index internally creates 11 sub tables (see
https://dev.mysql.com/doc/refman/8.4/en/innodb-fulltext-index.html) so
actually we have 192 tables in total here.

The search_index table name is generated dynamically based on the
account_id.
This commit is contained in:
Donal McBreen
2025-11-12 12:48:21 +00:00
committed by Mike Dalessio
parent 996140e053
commit 3aa4a1a562
13 changed files with 311 additions and 62 deletions
+4 -2
View File
@@ -1,8 +1,10 @@
namespace :search do
desc "Reindex all cards and comments in the search index"
task reindex: :environment do
puts "Clearing search index..."
ActiveRecord::Base.connection.execute("DELETE FROM search_index")
puts "Clearing search index shards..."
16.times do |i|
ActiveRecord::Base.connection.execute("DELETE FROM search_index_#{i}")
end
puts "Reindexing cards..."
Card.find_each do |card|