The search functionality was silently dropping CJK characters because:
1. Query sanitization used `\w` which only matches ASCII word characters
2. Stemmer split by whitespace, which doesn't work for CJK languages
3. Highlighter used `\b` word boundaries that don't apply to CJK
This commit fixes all three issues:
- Query: Use `\p{L}\p{N}` (Unicode letters/numbers) instead of `\w`
- Stemmer: Preserve CJK characters as-is without stemming, since CJK
languages don't have stemming rules like English
- Highlighter: Skip word boundary matching for CJK terms
Also extracts `CJK_PATTERN` to `Search` module to avoid duplication.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
In this case requests won't be performed from a secure context [1] and
the browser won't send the Sec-Fetch-Site header. This means non-GET
requests will be rejected because CSRF protection will fail.
With this change, we allow these requests with missing Sec-Fetch-Site
headers if:
- They happen over HTTP
- The app is not configured to force SSL
The Origin check happens in any case.
[1] https://developer.mozilla.org/en-US/docs/Web/Security/Defenses/Secure_Contexts#potentially_trustworthy_origins
The SSRF filter checked ipv4_mapped? but not ipv4_compat?, allowing
addresses like ::169.254.169.254 to bypass the link-local check and
reach cloud metadata endpoints.
Changes:
- Add ipv4_compat? check to block deprecated IPv4-compatible format
- Rename private_address? to blocked_address? (more accurate - method
blocks more than just RFC 1918 private ranges)
- Add IPv6 test coverage for both mapped and compat formats
Both ipv4_mapped and ipv4_compat formats are blocked entirely as
defense-in-depth: DNS never returns these formats, so they only
appear in attack scenarios.
HackerOne: #3481701
Instead, let's use a Loofah scrubber which will create DOM nodes
directly. This should be faster and is a tiny bit simpler, as well as
removing a potential HTML injection vector.
Also, add "noreferrer" to all `mailto:` links (already present on URLs).
Avatars are purposely accessible without authentication
(5e3b5b6d7c) because they can be in public
collections. Trying to restrict this by checking whether they're in fact
present in some public collection is rather expensive, so let's keep
them public.
- Update `Card::Closeable` to destroy associated "not now" state during close.
- Add new tests to verify closure behavior for different column types, including "not now".
The included SearchTestHelper disables transactional tests and has a specific setup that does not use fixtures.
This test however, ignores the setup and uses fixtures directly by creating comments and publishing cards making test state unpredictable.
By using the records set by the helper instead of fixtures, the state of the system is not polluted.
* Storage: harden reconcile for concurrent writes and fix board transfer
Reconcile now uses two-cursor approach: captures cursor before and after
the storage scan, aborting if they differ (entries added during scan).
Job retries 3x with 1-minute waits and limits concurrency to 1 per owner.
Board transfer now correctly moves storage for card description embeds
and comment embeds, not just direct attachments. Uses batched queries
to handle cards with thousands of comments efficiently.
Also fixes N+1 queries in attachment grouping via lookup maps.
* Storage: fix per-attachment reconcile and enforce no blob reuse
The storage ledger tracks per-attachment (not per-blob) as a business
abstraction for quotas. This fixes reconcile to match that model and
adds enforcement to prevent blob reuse in tracked contexts.
* Reconcile now uses joins(:blob).sum() for per-attachment counting
* New validation prevents reusing blobs across tracked attachments
* Race-safe storage_total creation with create_or_find_by
* Job efficiency: skip find_by when object already available
* Backfill script checks per-attachment, not just per-blob
We had a call about this. In short, we could reuse access tokens but then the user would see access tokens for every mobile device they have without any indication as to what is going on. So, since this really is just logging in instead of an integration which seems to be the primary purpose of access tokens, we can just use our regular session cookie for authentication.
As the tests for it could lead to confusion where it seems drafted cards
are not accessible to someone with access to the board and with the
direct drafted card URL.
replacing normalize_base36. This should make it easier to convert
UUIDs between hex and base36 formats, useful when debugging production
logs that contain hex UUIDs.
For example, if in a log you see:
User Load (0.3ms) SELECT `users`.* FROM `users` WHERE
`users`.`account_id` = x'019afab815897a4f920f3a24fab75400' AND
`users`.`id` = x'019afab815a2790688c58c7f8326e700' LIMIT 1
and you want to find that account or that user in the console, you can
now do:
id = ActiveRecord::Type::Uuid.hex_to_base36("019afab815897a4f920f3a24fab75400")
=> "03f6bilrvt3oghzhurll4pp8g"
Addresses an issue where User#familiar_name assumed `name` was always
present, potentially raising an exception during view rendering. Now
User validates name presence, and User#familiar_name handles blank
strings without error, in case any existing invalid records exist.
LU