Jeremy Daer
761d0b4a76
An append-only storage ledger replaces deadlock-prone synchronous counter updates and drift-prone async updates. All content storage (card images, card/comment/board description embeds) is tracked. Account exports (which expire) and avatars are not tracked. Storage is accounted for by Account and Board. Both consume the same event stream independently: no bubble-up storage bumps triggering deadlocks due to lock sequencing. Each calculates its own total from the underlying ledger with independent cursors and materialization. * Storage::Entry: Append-only ledger recording attach/detach/transfer events with delta bytes. Single event stream indexed for both Account and Board cursor queries. * Storage::Total: Polymorphic snapshot cache with cursor (last_entry_id) tracking which entries have been materialized. * Storage::Totaled concern: Provides bytes_used (fast snapshot) and bytes_used_exact (snapshot + pending) query modes, plus materialize_storage! to roll up pending entries. * Storage::Tracked concern: For models owning attachments (Card, Comment, Board). Provides board_for_storage_tracking for models where board is determined differently (Board returns self). Handles board transfers by recording transfer_out/transfer_in entries. * Storage::AttachmentTracking: Hooks ActiveStorage::Attachment lifecycle to record attach/detach entries. Handles ActionText::RichText embeds by traversing to the actual model. Snapshots context in before_destroy to handle cascading deletes where parent record may be gone by after_destroy_commit. * MaterializeJob: Rolls up pending entries into snapshot. Concurrency limited per owner to prevent duplicate work. * ReconcileJob: On-demand reconciliation against actual attachment storage for support/debugging. Compares ledger total to real bytes from card images, card embeds, comment embeds, and board embeds. Usage: * account.bytes_used / board.bytes_used: fast, slightly stale bytesize * account.bytes_used_exact / board.bytes_used_exact: real-time bytesize * Storage::Entry: audit trail for debugging and point-in-time queries
57 lines
1.7 KiB
Ruby
57 lines
1.7 KiB
Ruby
module Board::Storage
|
|
extend ActiveSupport::Concern
|
|
include Storage::Totaled
|
|
|
|
# Board's own embeds (public_description) count toward itself
|
|
def board_for_storage_tracking
|
|
self
|
|
end
|
|
|
|
private
|
|
BATCH_SIZE = 1000
|
|
|
|
# Calculate actual storage by summing blob sizes.
|
|
#
|
|
# Uses batched pluck queries to avoid loading huge ID arrays, and avoids
|
|
# ActiveRecord model queries on ActiveStorage tables to sidestep cross-pool
|
|
# issues when ActiveStorage uses separate connection pools (e.g., with replicas).
|
|
def calculate_real_storage_bytes
|
|
card_image_bytes + card_embed_bytes + comment_embed_bytes + board_embed_bytes
|
|
end
|
|
|
|
def card_image_bytes
|
|
sum_blob_bytes_in_batches \
|
|
ActiveStorage::Attachment.where(record_type: "Card", name: "image"),
|
|
cards.pluck(:id)
|
|
end
|
|
|
|
def card_embed_bytes
|
|
sum_embed_bytes_for "Card", cards.pluck(:id)
|
|
end
|
|
|
|
def comment_embed_bytes
|
|
sum_embed_bytes_for "Comment", Comment.where(card_id: cards.pluck(:id)).pluck(:id)
|
|
end
|
|
|
|
def board_embed_bytes
|
|
sum_embed_bytes_for "Board", [ id ]
|
|
end
|
|
|
|
def sum_embed_bytes_for(record_type, record_ids)
|
|
rich_text_ids = ActionText::RichText \
|
|
.where(record_type: record_type, record_id: record_ids)
|
|
.pluck(:id)
|
|
|
|
sum_blob_bytes_in_batches \
|
|
ActiveStorage::Attachment.where(record_type: "ActionText::RichText", name: "embeds"),
|
|
rich_text_ids
|
|
end
|
|
|
|
def sum_blob_bytes_in_batches(base_scope, record_ids)
|
|
record_ids.each_slice(BATCH_SIZE).sum do |batch_ids|
|
|
blob_ids = base_scope.where(record_id: batch_ids).pluck(:blob_id)
|
|
ActiveStorage::Blob.where(id: blob_ids).sum(:byte_size)
|
|
end
|
|
end
|
|
end
|