Commit Graph

281 Commits

Author SHA1 Message Date
Stanko K.R. 19ae555c2e Convert Account::SingleUserExport to User::DataExport 2026-02-02 12:36:48 +01:00
Stanko K.R. 8953920652 Implement full account exports 2026-02-02 12:36:48 +01:00
Stanko K.R. 6f17e3867d Split Export model into user and whole-account export 2026-02-02 12:36:31 +01:00
Mike Dalessio 1570c16f67 Make Reaction polymorphic
Add migration to replace comment_id with polymorphic reactable
association (reactable_type, reactable_id), enabling reactions on both
comments and cards.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-22 12:29:54 -05:00
Stanko Krtalić eceb6c9b43 Add self-service account deletion (#2246)
* Add self-service account deletion

* Disable access to cancelled accounts & implement Stripe interactions

* Add tests

* Fix failing tests

* Remove cancelled accounts from lists

* Fix incorrect redirect

* Use _path instead of _url for consistency

* Don't track how far the inicieration job got

We still want the step tracking so that the job can be interrupted. But, since the scope is idempotent, we don't need to track how far it got.

* Specify the exact time when the data will be deleted

* Fix crash due to unadvancable cursor

* Rename up_for_incineration to due_for_incineration

* Regenrate the schema

* Fix incorrect path check

* Migrate the SQLite schema

* Only show the cancel button on cancellable accounts

* Check that a subscirption method exists before calling it

* Ignore job failures due to missing records when an account gets deleted

* Skip sending notifications on cancelled accounts

* Use collbacks to integrate

* Add a blank line between queue_as and discard_on

* Break checks into methods

* Inline methods

* Rename Account::IncinerateJob

* Run migrations

* Migrate SQLite
2026-01-12 14:21:05 +01:00
Jorge Manrubia 6d05ab428e Remove engagements
We are not using this anymore!
2026-01-07 16:43:33 +01:00
Jeremy Daer 761d0b4a76 Speedy, auditable, deadlock-resistant storage tracking (#2026)
An append-only storage ledger replaces deadlock-prone synchronous
counter updates and drift-prone async updates.

All content storage (card images, card/comment/board description embeds)
is tracked. Account exports (which expire) and avatars are not tracked.

Storage is accounted for by Account and Board. Both consume the same
event stream independently: no bubble-up storage bumps triggering
deadlocks due to lock sequencing. Each calculates its own total from
the underlying ledger with independent cursors and materialization.

* Storage::Entry: Append-only ledger recording attach/detach/transfer
  events with delta bytes. Single event stream indexed for both
  Account and Board cursor queries.

* Storage::Total: Polymorphic snapshot cache with cursor (last_entry_id)
  tracking which entries have been materialized.

* Storage::Totaled concern: Provides bytes_used (fast snapshot) and
  bytes_used_exact (snapshot + pending) query modes, plus
  materialize_storage! to roll up pending entries.

* Storage::Tracked concern: For models owning attachments (Card,
  Comment, Board). Provides board_for_storage_tracking for models
  where board is determined differently (Board returns self).
  Handles board transfers by recording transfer_out/transfer_in entries.

* Storage::AttachmentTracking: Hooks ActiveStorage::Attachment lifecycle
  to record attach/detach entries. Handles ActionText::RichText embeds
  by traversing to the actual model. Snapshots context in before_destroy
  to handle cascading deletes where parent record may be gone by
  after_destroy_commit.

* MaterializeJob: Rolls up pending entries into snapshot. Concurrency
  limited per owner to prevent duplicate work.

* ReconcileJob: On-demand reconciliation against actual attachment
  storage for support/debugging. Compares ledger total to real bytes
  from card images, card embeds, comment embeds, and board embeds.

Usage:
* account.bytes_used / board.bytes_used: fast, slightly stale bytesize
* account.bytes_used_exact / board.bytes_used_exact: real-time bytesize
* Storage::Entry: audit trail for debugging and point-in-time queries
2025-12-10 16:04:30 -08:00
David Heinemeier Hansson 895b0e13b8 Drop the need for access tokens to have a session 2025-12-10 09:23:52 +01:00
David Heinemeier Hansson db29562c4c Tie access token directly to session
We need to present them differently in the session list and prevent them
from being deleted
2025-12-10 09:23:52 +01:00
David Heinemeier Hansson a81c681a8d Add access token authentication via HTTP AUTHORIZATION bearer header 2025-12-10 09:23:52 +01:00
Mike Dalessio 4602cd3cdd Add verified_at timestamp to use for spam prevention
User are marked as verified after a join code is redeemed. The user is
redirected to Users::VerificationsController, either:

- after submitting a valid magic link code,
- or immediately after redeeming the join code (if they're already
  authenticated with the correct identity)

Account owners are automatically verified when the account is
created (because they have already provided a magic link code at that
point).

This sets up for later commits that will backfill existing users and
require verification before sending notification emails.

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-05 21:51:44 -05:00
Kevin McConnell e16cc21b0a Add "data export" feature
- Adds a button in Account Settings where you can request a ZIP export of your
  Fizzy data
- Export files are created in the background. When ready, a link to
  download them is sent to the requester.
- Exports expire after 24 hours. And are limited to 10 per day.
2025-12-01 15:23:26 +00:00
Mike Dalessio edf6b53469 Introduce an "owner" role, and prevent it from being administered
to prevent the owner from being demoted or kicked out.

ref: https://app.fizzy.do/5986089/cards/3213
2025-11-29 14:13:29 -05:00
Stanko K.R. 2311efd03e Make sign up Magic Links work across devices
Previously if someone started signing in on their phone, but finished it on their laptop, they'd end up on the menu screen with no account. Bu tracking the purpose of a Magic Link we can always direct the user to sign up if they requested a magic link through sign up. This also makes the logic for changing the copy in the email more robust.
2025-11-29 12:36:00 +01:00
Jorge Manrubia 4e09352c09 Bring simple signup flow from the fizzy-saas gem
We skip the QB code and we fill external account ids automatically on creation with a sequence

See:
https://github.com/basecamp/fizzy-saas/pull/7
2025-11-28 15:53:58 +01:00
Jorge Manrubia 2061ab4ab2 Use dedicated sequence record instead of deal with issues with ids and uuids when using the previous approach 2025-11-28 15:53:58 +01:00
Jorge Manrubia 07b4c55185 Auto-increment external_account_id automatically by default to allow upcoming simple signups 2025-11-28 15:53:58 +01:00
Donal McBreen c4498212dc Merge branch 'main' into sqlite
* main: (116 commits)
  Ensure avatar thumbnails are square
  Update useragent to recognize twitterbot/facebot
  Add defensive styles for non-square avatar images
  Update test for copy changes
  Missed commit
  AI: standardize on https://agents.md
  Make it clear this is just notifications, not comprehensive activity
  AI: configure MCP servers for Chrome, Grafana, and Sentry (#1727)
  Allow requests from Google Image Proxy
  Update to basecamp's useragent fork
  Clean up a little bit the CSRF reporting code
  Claude: production observability guidance (#1725)
  Prevent autoscroll to the root columns container to prevent jump on page load
  Include full name string so you can type your name to filter
  Prioritize current user and assigned users in assignment dropdown
  Check and report on Sec-Fetch-Site header for forgery protection
  bundle update
  Bump bootsnap from 1.18.6 to 1.19.0
  Bump rails from `077c3ad` to `17f6e00`
  Fix cards getting stuck in edit mode
  ...
2025-11-26 10:07:41 +00:00
Donal McBreen 67f648f356 Enforce MySQL string/text limits in SQLite
To ensure consistent column lengths, we'll add limits to string and text
columns which match MySQL defaults.
2025-11-25 14:50:40 +00:00
Stanko K.R. f64a49dcc0 Determine staff members based on flag
This replaces the old system based on email addresses, the aim is to make it more generic and thus appropriate for OSS/self-hosting

See: https://3.basecamp.com/2914079/buckets/37331921/todos/9302705265#__recording_9320051455
2025-11-25 14:09:31 +01:00
Stanko K.R. cc49a1b772 Allow long user agent strings
Copied this from BC and HEY which also use a string with a 4096 character length

See: https://app.fizzy.do/5986089/cards/3066
2025-11-25 12:09:02 +01:00
Donal McBreen e76f159f05 Gate search_record shard migrations on adapter type 2025-11-21 11:44:37 +00:00
Donal McBreen 366dbd393c Merge branch 'main' into sqlite 2025-11-21 11:39:47 +00:00
Donal McBreen cb71af8231 Remove the old fulltext indexes
We only search by account_key, content, and title now.
2025-11-21 11:25:16 +00:00
Donal McBreen 74d1cf735b Only add and fill account_key 2025-11-21 10:20:27 +00:00
Donal McBreen 70dd754cf7 Account key for search records
Add an account key field to improve search performance. This field
allows us to filter the records by account directly in the fulltext
index, so we only need to examine rows belonging to the relevant account.
2025-11-21 09:38:35 +00:00
Donal McBreen 5edb0a2075 Update schema 2025-11-21 09:15:19 +00:00
Donal McBreen bb36b4846f Support SQLite searching 2025-11-21 09:15:19 +00:00
Mike Dalessio 1bbb6d0c03 Fix race condition in Card::ActivitySpike::DetectionJob
The check-then-act pattern in `register_activity_spike` has been
replaced with `find_or_create_by!` to eliminate the race condition
that could lead to creating multiple activity spikes for a card. To
support this change, the `card_id` index on `cards_activity_spikes`
has been made unique.

ref: https://app.fizzy.do/5986089/cards/3063
2025-11-20 15:40:42 -05:00
Mike Dalessio fb742d6d29 Make sure the migration to remove FK constraints is robust
where we've already removed the constraints.
2025-11-20 15:16:26 -05:00
Mike Dalessio 6e67a7787a Drop all foreign key constraints
These have been a contributing cause to some deadlocks, but also this
makes some operational tasks harder (data loading, data truncation)
and has some performance impact on insert/update/delete operations.

The team consensus is that we aren't relying on them, and so we're
comfortable with the referential integrity risk.
2025-11-20 15:01:25 -05:00
Stanko K.R. ab07488fbd Fix int and string overflow during prod import 2025-11-18 16:16:01 +01:00
Stanko K.R. b7404087bd Fix schema incomatibilities with production 2025-11-17 20:31:42 +01:00
Stanko K.R. d915bff326 Fix admin having duplicate user after import 2025-11-17 09:12:41 -05:00
Donal McBreen 067e889cb4 Make migrations reversible 2025-11-17 09:12:40 -05:00
Donal McBreen ea970b195a Drop Search::Results table 2025-11-17 09:12:40 -05:00
Donal McBreen 28efe28f24 Replace Search::Index with Search::Records
Lean on ActiveRecord models for searching and strip out the raw SQL.
Replaces the search_index_* tables with sharded search_records_* tables
as that allows us to use a Search::Record model name.

A Class is dynamically created for each record table shard so that we
and we can access it via the Search::Record.for_account(account_id)
method.
2025-11-17 09:12:40 -05:00
Mike Dalessio fe6df7085f Finish the rollout of account_id to all core data models
Schema:
- add account_id to tables it was missing from
- make account_id a required column everywhere
- add [account_id] indexes, or add `account_id` to existing indices

Models:
- add `belongs_to :account` to all models (default to using a domain
  model's account whenever possible)
- add account_id in all the necessary fixtures
- add account_id to insert_all hashes
- pass account_id to a few initialize calls

Miscellaneous:
- update the import script to set account_id

Note that I'm not adding account_id to the join tables primarily
because I couldn't think of an easy way to populate it without making
it a full Join model, and that was more work than I have time to take
on right now.
2025-11-17 09:12:40 -05:00
Stanko K.R. edf837fed3 Drop memberships 2025-11-17 09:12:39 -05:00
Donal McBreen f4729577b6 Use uuids in the search_index tables too 2025-11-17 09:12:35 -05:00
Donal McBreen fc56ad9d7c Combine uuid-old and uuid branch changes
- Switch to binary 16 for UUID keys
- Remove AccountScopedRecord base class, all model use binary uuids now
- Fix the search sql to serialize uuids properly
- Patch the MySQL schema dumper to output binary lengths
2025-11-17 09:12:34 -05:00
Kevin McConnell 6c04b56f4b Compact the UUIDs to base36 2025-11-17 09:12:31 -05:00
Mike Dalessio e4011ef211 Update primary keys on customer data to UUIDs
- schema changes to primary and foreign keys
- fixture changes
- customer data models subclass AccountScopedRecord
- import script updated
2025-11-17 09:12:30 -05:00
Stanko K.R. 6858b96619 Simplify join codes 2025-11-17 09:12:17 -05:00
Donal McBreen 3aa4a1a562 Shard the search index into 16 tables
Create search_index_0 to search_index_15 tables and shard each index by
account id. MySQL has no ability to pre-filter fulltext indexes by
another field so this is the best bet for improving performance.

Each fulltest index internally creates 11 sub tables (see
https://dev.mysql.com/doc/refman/8.4/en/innodb-fulltext-index.html) so
actually we have 192 tables in total here.

The search_index table name is generated dynamically based on the
account_id.
2025-11-17 09:12:17 -05:00
Donal McBreen e8abc66eba MySQL search
Add a single search_index table for full-text search of cards and
comments.

For the search there is a full-text index on the title and content
columns. The board_ids is also included in the table and accessible
board ids are pre-loaded and included in the search query. This allows
us to filter out inaccessible records before joining with other tables.

Right now the search is just using boolean search. This would give us
a bunch of syntax options
(see https://dev.mysql.com/doc/refman/8.4/en/fulltext-boolean.html)
except the search query filters those out.

I've removed the searchable_by method for now - everything is built
on the assumption that there's a single search index table and all data
must fit into it.

Queries are written in SQL, we don't have a SearchIndex ActiveRecord
model. That's because we'll likely want to shard the table and it will
be simpler to just keep with the raw SQL for that.

There's no stemming, highlighting or snippet extraction yet - we are
dumping the full description in the search results.

Data can be reindexed with the search:reindex rake task.
2025-11-17 09:12:17 -05:00
Stanko K.R. 32d96db7d9 Replace card id with card number 2025-11-17 09:12:15 -05:00
Kevin McConnell 4e44cd5444 Flatten schema migrations 2025-11-17 09:11:57 -05:00
Stanko K.R. 8b1ca5e7fb Remove unused account associations 2025-11-17 09:11:42 -05:00
Stanko K.R. ba3f6b188e Add account_id to everything 2025-11-17 09:11:42 -05:00