Commit Graph

344 Commits

Author SHA1 Message Date
Stanko K.R. 2311efd03e Make sign up Magic Links work across devices
Previously if someone started signing in on their phone, but finished it on their laptop, they'd end up on the menu screen with no account. Bu tracking the purpose of a Magic Link we can always direct the user to sign up if they requested a magic link through sign up. This also makes the logic for changing the copy in the email more robust.
2025-11-29 12:36:00 +01:00
Jorge Manrubia 4e09352c09 Bring simple signup flow from the fizzy-saas gem
We skip the QB code and we fill external account ids automatically on creation with a sequence

See:
https://github.com/basecamp/fizzy-saas/pull/7
2025-11-28 15:53:58 +01:00
Jorge Manrubia 2061ab4ab2 Use dedicated sequence record instead of deal with issues with ids and uuids when using the previous approach 2025-11-28 15:53:58 +01:00
Jorge Manrubia 07b4c55185 Auto-increment external_account_id automatically by default to allow upcoming simple signups 2025-11-28 15:53:58 +01:00
Mike Dalessio b78977f563 Make db/seeds.rb rerunnable when I want to add more cards. 2025-11-27 10:46:48 -05:00
Donal McBreen a5fc6eeaec Update SQLite schema 2025-11-26 10:18:55 +00:00
Donal McBreen c4498212dc Merge branch 'main' into sqlite
* main: (116 commits)
  Ensure avatar thumbnails are square
  Update useragent to recognize twitterbot/facebot
  Add defensive styles for non-square avatar images
  Update test for copy changes
  Missed commit
  AI: standardize on https://agents.md
  Make it clear this is just notifications, not comprehensive activity
  AI: configure MCP servers for Chrome, Grafana, and Sentry (#1727)
  Allow requests from Google Image Proxy
  Update to basecamp's useragent fork
  Clean up a little bit the CSRF reporting code
  Claude: production observability guidance (#1725)
  Prevent autoscroll to the root columns container to prevent jump on page load
  Include full name string so you can type your name to filter
  Prioritize current user and assigned users in assignment dropdown
  Check and report on Sec-Fetch-Site header for forgery protection
  bundle update
  Bump bootsnap from 1.18.6 to 1.19.0
  Bump rails from `077c3ad` to `17f6e00`
  Fix cards getting stuck in edit mode
  ...
2025-11-26 10:07:41 +00:00
Donal McBreen 97bcdf1853 Enforce column limits via concern
SQLite columns lengths are purely informational, so we'll enforce the
limits via a concern that checks the lengths from the schema.
2025-11-25 15:32:34 +00:00
Donal McBreen 67f648f356 Enforce MySQL string/text limits in SQLite
To ensure consistent column lengths, we'll add limits to string and text
columns which match MySQL defaults.
2025-11-25 14:50:40 +00:00
Stanko K.R. f64a49dcc0 Determine staff members based on flag
This replaces the old system based on email addresses, the aim is to make it more generic and thus appropriate for OSS/self-hosting

See: https://3.basecamp.com/2914079/buckets/37331921/todos/9302705265#__recording_9320051455
2025-11-25 14:09:31 +01:00
Stanko K.R. cc49a1b772 Allow long user agent strings
Copied this from BC and HEY which also use a string with a 4096 character length

See: https://app.fizzy.do/5986089/cards/3066
2025-11-25 12:09:02 +01:00
Donal McBreen b92aba383e account_key is MySQL specific 2025-11-21 11:47:28 +00:00
Donal McBreen e76f159f05 Gate search_record shard migrations on adapter type 2025-11-21 11:44:37 +00:00
Donal McBreen 366dbd393c Merge branch 'main' into sqlite 2025-11-21 11:39:47 +00:00
Donal McBreen cb71af8231 Remove the old fulltext indexes
We only search by account_key, content, and title now.
2025-11-21 11:25:16 +00:00
Donal McBreen 74d1cf735b Only add and fill account_key 2025-11-21 10:20:27 +00:00
Donal McBreen 70dd754cf7 Account key for search records
Add an account key field to improve search performance. This field
allows us to filter the records by account directly in the fulltext
index, so we only need to examine rows belonging to the relevant account.
2025-11-21 09:38:35 +00:00
Donal McBreen b4e3d08689 Add long back to dumped schema, need to check why it wants to change it though 2025-11-21 09:15:19 +00:00
Donal McBreen 5edb0a2075 Update schema 2025-11-21 09:15:19 +00:00
Donal McBreen bb36b4846f Support SQLite searching 2025-11-21 09:15:19 +00:00
Mike Dalessio 1bbb6d0c03 Fix race condition in Card::ActivitySpike::DetectionJob
The check-then-act pattern in `register_activity_spike` has been
replaced with `find_or_create_by!` to eliminate the race condition
that could lead to creating multiple activity spikes for a card. To
support this change, the `card_id` index on `cards_activity_spikes`
has been made unique.

ref: https://app.fizzy.do/5986089/cards/3063
2025-11-20 15:40:42 -05:00
Mike Dalessio fb742d6d29 Make sure the migration to remove FK constraints is robust
where we've already removed the constraints.
2025-11-20 15:16:26 -05:00
Mike Dalessio 6e67a7787a Drop all foreign key constraints
These have been a contributing cause to some deadlocks, but also this
makes some operational tasks harder (data loading, data truncation)
and has some performance impact on insert/update/delete operations.

The team consensus is that we aren't relying on them, and so we're
comfortable with the referential integrity risk.
2025-11-20 15:01:25 -05:00
Kevin McConnell b63ec319a2 Remove trailing newline (appease Rubocop) 2025-11-18 18:44:45 +00:00
Mike Dalessio 0e5d320045 Skip seeds in non-development environments
because we need to bootstrap the database in production
2025-11-18 11:54:35 -05:00
Stanko K.R. ab07488fbd Fix int and string overflow during prod import 2025-11-18 16:16:01 +01:00
Stanko K.R. b7404087bd Fix schema incomatibilities with production 2025-11-17 20:31:42 +01:00
Stanko K.R. d915bff326 Fix admin having duplicate user after import 2025-11-17 09:12:41 -05:00
Donal McBreen 067e889cb4 Make migrations reversible 2025-11-17 09:12:40 -05:00
Donal McBreen ea970b195a Drop Search::Results table 2025-11-17 09:12:40 -05:00
Donal McBreen 28efe28f24 Replace Search::Index with Search::Records
Lean on ActiveRecord models for searching and strip out the raw SQL.
Replaces the search_index_* tables with sharded search_records_* tables
as that allows us to use a Search::Record model name.

A Class is dynamically created for each record table shard so that we
and we can access it via the Search::Record.for_account(account_id)
method.
2025-11-17 09:12:40 -05:00
Mike Dalessio fe6df7085f Finish the rollout of account_id to all core data models
Schema:
- add account_id to tables it was missing from
- make account_id a required column everywhere
- add [account_id] indexes, or add `account_id` to existing indices

Models:
- add `belongs_to :account` to all models (default to using a domain
  model's account whenever possible)
- add account_id in all the necessary fixtures
- add account_id to insert_all hashes
- pass account_id to a few initialize calls

Miscellaneous:
- update the import script to set account_id

Note that I'm not adding account_id to the join tables primarily
because I couldn't think of an easy way to populate it without making
it a full Join model, and that was more work than I have time to take
on right now.
2025-11-17 09:12:40 -05:00
Mike Dalessio 7b1b5f464d Fix up seeds to work in a post-membership world. 2025-11-17 09:12:40 -05:00
Stanko K.R. edf837fed3 Drop memberships 2025-11-17 09:12:39 -05:00
Mike Dalessio 7f2bd9a636 re-sync db/schema.rb 2025-11-17 09:12:36 -05:00
Donal McBreen 6323e0abee Use uuid as the id type in schema.rb
This matches the format in the migration files and makes the type_to_sql
override redundant.

The only wart we are left with is that since there are no true UUID
types in MySQL we'll have to assume that binary(16) columns are UUIDs.

In practice this is probably fine as the MySQL adapter doesn't map any
other types to binary.
2025-11-17 09:12:36 -05:00
Donal McBreen f4729577b6 Use uuids in the search_index tables too 2025-11-17 09:12:35 -05:00
Donal McBreen fc56ad9d7c Combine uuid-old and uuid branch changes
- Switch to binary 16 for UUID keys
- Remove AccountScopedRecord base class, all model use binary uuids now
- Fix the search sql to serialize uuids properly
- Patch the MySQL schema dumper to output binary lengths
2025-11-17 09:12:34 -05:00
Donal McBreen 05a7ec84e2 Make search tables work with UUID references 2025-11-17 09:12:31 -05:00
Donal McBreen e3064ec6ad Fix merge 2025-11-17 09:12:31 -05:00
Kevin McConnell 6c04b56f4b Compact the UUIDs to base36 2025-11-17 09:12:31 -05:00
Mike Dalessio e4011ef211 Update primary keys on customer data to UUIDs
- schema changes to primary and foreign keys
- fixture changes
- customer data models subclass AccountScopedRecord
- import script updated
2025-11-17 09:12:30 -05:00
Stanko K.R. 6858b96619 Simplify join codes 2025-11-17 09:12:17 -05:00
Donal McBreen 3aa4a1a562 Shard the search index into 16 tables
Create search_index_0 to search_index_15 tables and shard each index by
account id. MySQL has no ability to pre-filter fulltext indexes by
another field so this is the best bet for improving performance.

Each fulltest index internally creates 11 sub tables (see
https://dev.mysql.com/doc/refman/8.4/en/innodb-fulltext-index.html) so
actually we have 192 tables in total here.

The search_index table name is generated dynamically based on the
account_id.
2025-11-17 09:12:17 -05:00
Donal McBreen e8abc66eba MySQL search
Add a single search_index table for full-text search of cards and
comments.

For the search there is a full-text index on the title and content
columns. The board_ids is also included in the table and accessible
board ids are pre-loaded and included in the search query. This allows
us to filter out inaccessible records before joining with other tables.

Right now the search is just using boolean search. This would give us
a bunch of syntax options
(see https://dev.mysql.com/doc/refman/8.4/en/fulltext-boolean.html)
except the search query filters those out.

I've removed the searchable_by method for now - everything is built
on the assumption that there's a single search index table and all data
must fit into it.

Queries are written in SQL, we don't have a SearchIndex ActiveRecord
model. That's because we'll likely want to shard the table and it will
be simpler to just keep with the raw SQL for that.

There's no stemming, highlighting or snippet extraction yet - we are
dumping the full description in the search results.

Data can be reindexed with the search:reindex rake task.
2025-11-17 09:12:17 -05:00
Stanko K.R. 32d96db7d9 Replace card id with card number 2025-11-17 09:12:15 -05:00
Kevin McConnell 4e44cd5444 Flatten schema migrations 2025-11-17 09:11:57 -05:00
Mike Dalessio 4ac87a9ad9 Fix seeds 2025-11-17 09:11:57 -05:00
Stanko K.R. 8b1ca5e7fb Remove unused account associations 2025-11-17 09:11:42 -05:00
Stanko K.R. ba3f6b188e Add account_id to everything 2025-11-17 09:11:42 -05:00