The prior commit added it to saas/Gemfile but the production Docker
build uses Gemfile.saas (via BUNDLE_GEMFILE). This ensures the XML
parameter parser is present in the actual runtime environment.
Queenbee sends account sync/cancel/etc requests as XML via
ActiveResource, including the queenbee_signature in the XML body.
Rails removed built-in XML parameter parsing in Rails 4.0, so without
this gem the XML body is silently ignored and the signature check
always fails with 403.
Follow-up to #2550 which added a created_at index and increased cleanup
frequency. The delete itself was still unbatched and vulnerable whenever
a backlog accumulates (first run after deploy, clock skew, etc).
Delete in small batches with a pause between each to let other
transactions through, following the SolidQueue pattern.
* Add index on webhook_deliveries.created_at
The stale scope (WHERE created_at < ?) and ordered scope
(ORDER BY created_at DESC) both benefit from this index.
Without it, Webhook::Delivery.cleanup does a full table scan.
* Run webhook delivery cleanup every 15 minutes
Each run now deletes ~15 minutes of newly-stale records instead
of ~4 hours worth, a 16x reduction in per-run write volume.
When concurrent NotifyRecipientsJobs process an Event and a Mention for
the same user+card, Rails' dirty tracking can skip writing source_type
in the UPDATE if it hasn't changed from the stale in-memory value,
leaving source_type and source_id mismatched (e.g. source_type='Event'
with a Mention's source_id, resulting in a nil source).
Force source_type to always be included in the UPDATE via
source_type_will_change! to prevent this.
Here's a sample timeline of this race condition happening in the real
world (with simplified IDs):
Two `NotifyRecipientsJob` involving notification `03fklpu`, same card,
same user, same comment — enqueued within 50ms of each other:
1. Both jobs load notification `03fklpu` — it has source_type='Mention'
(from a previous job)
2. `EventNotifier` writes at 21:22:36.051: ```sql SET
source_type='Event', source_id=<event_id>, unread_count=1 ```
3. `source_type` included because it changed ('Mention' → 'Event')
4. `MentionNotifier` writes at 21:22:36.057 (~6ms later): ```sql SET
source_id=<mention_id>, unread_count=1 ```
5. No `source_type`! It was `'Mention'` when loaded and `'Mention'` is
what it's setting → not dirty → skipped
6. Final DB state: `source_type='Event'` (from step 2, untouched),
`source_id=<mention_id>` (from step 3)
`Notification.source` now does `Event.find(<mention_id>) → nil.`
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The convert_gids_to_sgids method resolved GIDs globally and minted
valid SGIDs without verifying the record belongs to the importing
account. This adds the same account_id check that the export path
already has, plus RecordNotFound handling for invalid GIDs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* main: (33 commits)
Revert "Add temporary performance degradation message"
Fix double-escaped HTML entities in webhook payloads
Fix crash when a notification is deleted while a bundle is being sent
Fix race condition on notification creation
Fix double _seconds suffix in GVL request wait metric name (#2539)
Add GVL contention metrics via gvltools (#2538)
Backfill tests for Event::Description strings' html-safety
Update copy
Add temporary performance degradation message
Render inline code in card titles (#2518)
Add stackprof for profiling
Table style tweaks
Fix dark mode colors
Index notifications on updated_at
Stack notifications everywhere
HR fixes
Removing duplicate code language picker
Regression fixes
Touch up link button hover styles
Remove link dialog size limit
...
Rename histogram from :request_wait_seconds to :request_wait so Yabeda's
unit: :seconds produces gvl_request_wait_seconds instead of
gvl_request_wait_seconds_seconds.
* Add GVL contention metrics via gvltools and Yabeda
Expose per-request and process-wide GVL wait time as Prometheus metrics
to diagnose suspected GVL contention from Solid Cable and Action Cable
threads in Puma workers.
Metrics: gvl_request_wait_seconds (histogram), gvl_waiting_threads
(gauge), gvl_global_timer_total_seconds (gauge).
* Add unit: :seconds to GVL histogram for consistency
Alp Keser