The search functionality was silently dropping CJK characters because:
1. Query sanitization used `\w` which only matches ASCII word characters
2. Stemmer split by whitespace, which doesn't work for CJK languages
3. Highlighter used `\b` word boundaries that don't apply to CJK
This commit fixes all three issues:
- Query: Use `\p{L}\p{N}` (Unicode letters/numbers) instead of `\w`
- Stemmer: Preserve CJK characters as-is without stemming, since CJK
languages don't have stemming rules like English
- Highlighter: Skip word boundary matching for CJK terms
Also extracts `CJK_PATTERN` to `Search` module to avoid duplication.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Schema:
- add account_id to tables it was missing from
- make account_id a required column everywhere
- add [account_id] indexes, or add `account_id` to existing indices
Models:
- add `belongs_to :account` to all models (default to using a domain
model's account whenever possible)
- add account_id in all the necessary fixtures
- add account_id to insert_all hashes
- pass account_id to a few initialize calls
Miscellaneous:
- update the import script to set account_id
Note that I'm not adding account_id to the join tables primarily
because I couldn't think of an easy way to populate it without making
it a full Join model, and that was more work than I have time to take
on right now.
- Switch to binary 16 for UUID keys
- Remove AccountScopedRecord base class, all model use binary uuids now
- Fix the search sql to serialize uuids properly
- Patch the MySQL schema dumper to output binary lengths
Jorge Manrubia