simply_based/SESSION_REPORT.md

# simply_couch → Couchbase: Session Report

**For:** Benjamin ter Kuile (bterkuile)
**Prepared by:** Greg (via Claude Code / Opus 4.8)
**Date:** 2026-06-20

This document summarizes a working session that started from one request —
"compartmentalize simply_couch's data backend so CouchDB could be swapped for
Couchbase" — and grew, through your input in the thread, into three concrete
artifacts plus an architectural recommendation. Everything below was built and
verified against a live database (CouchDB 3.5.2 and Couchbase Community 7.x in
local Docker); nothing here is hypothetical.

---

## TL;DR

Three artifacts, in increasing ambition:

1. **PR #10 on `bterkuile/simply_couch`** — a behaviour-preserving refactor that
   turns the leaky single-driver seam into a real, documented adapter contract.
   Full suite green (453 examples). CouchDB still the default.
2. **`simply_couch-couchbase`** — a Couchbase *adapter plugin* for the refactored
   simply_couch (kept the CouchDB-shaped API). SDK-based, N1QL view translation.
3. **`simply_based`** — the *revolutionary* path you asked for: a standalone
   Couchbase-native ORM. `include SimplyBased::Model`, no CouchDB concepts left.
   Runs the simply_couch test suite: **403/403 applicable examples green**.

And a candid recommendation at the end: whether a Couchbase-native ORM is
actually the right call versus a standard Postgres + Redis Rails stack.

---

## The core strategy point: Couchbase ≠ CouchDB

Despite the shared name and lineage (Damien Katz), the two are not API-compatible,
and a literal `simply_couchbase_rest` drop-in for `couchrest` cannot exist:

| Concern | CouchDB (couchrest) | Couchbase |
|---|---|---|
| Data path | HTTP REST per-document | binary KV via SDK (REST is management-only) |
| Query | JS map/reduce **design-doc views** | **N1QL / SQL++** + GSI (CB views deprecated in 7.0) |
| Concurrency | MVCC `_rev`, conflict docs | CAS (uint64), no user-facing revisions |
| Bulk | `_bulk_docs` | batched KV / subdoc ops |
| Attachments | native blob attachments | none — store externally (S3/local) |
| "CouchDB interface" | — | only the deprecated Views engine (port 8092 CAPI), or Sync Gateway's replication protocol — neither is a general server API for an ORM |

So Couchbase is reached the way it's meant to be: the SDK for KV, **N1QL** for
queries. Not a REST shim. The pinch point is the view system — but simply_couch
builds views from **structured Ruby spec objects**, not ad-hoc JS, so the finder
and association queries translate deterministically to N1QL. That's what made the
whole thing tractable.

---

## Artifact 1 — PR #10: the adapter-contract refactor

**Link:** https://github.com/bterkuile/simply_couch/pull/10 (open; you were
notified on open).

**What it does (behaviour-preserving — zero functional change for couchrest users):**

- Adds `SimplyCouch::Adapter` — the backend-neutral contract the model layer
  depends on (`save_document`, `load_document`, `destroy_document`, `bulk_*`,
  `view`, `first`, `create_database!`, `drop_database!`), plus an exception
  contract (drivers translate their errors to `SimplyCouch::Conflict` /
  `NotFound`).
- Moves the existing CouchDB code out of `Model::DatabaseInstance` into
  `SimplyCouch::Adapters::CouchRest` **verbatim** (a `DatabaseInstance` alias is
  kept). This is a move, not a rewrite.
- **Lazy driver loading:** `require 'simply_couch'` now loads with no driver
  present; couchrest loads only when its adapter is first used.
- Adapter registry: `SimplyCouch.register_adapter(name, class, require_path:)`
  + `SimplyCouch.adapter = :name`.
- Stops the model/association/view layers reaching through
  `database.couchrest_database` (5 sites → the neutral `database.bulk_load`).
- `SimplyCouch::DocumentCodec` — rebuilds a model from a plain hash via the
  `ruby_class` tag, so non-couchrest backends can hydrate documents.
- Exposes the view-spec `klass`/`options` as the backend-neutral query IR.
- Hardens the load guard so a plugin gem can open `module SimplyCouch` for its
  own namespace before the library loads.
- Adds shared `'a simply_couch adapter'` RSpec examples (reusable parity proof).
- Version 0.1.2 → 0.2.0; README/CHANGELOG updated.

**Verification:** 453 examples, 0 failures vs a live CouchDB 3.5.2. No model-facing
signature changed. (Two pre-existing flaky DB-reset races predate the PR.)

---

## Artifact 2 — `simply_couch-couchbase` (adapter plugin)

A separate gem that plugs the **refactored** simply_couch into Couchbase while
keeping the CouchDB-shaped API. `SimplyCouch::Adapters::Couchbase` on the official
`couchbase` SDK: KV for documents, CAS↔`_rev`, and an `N1qlTranslator` that
renders the structured `ModelViewSpec` / `DeletedModelViewSpec` (all_documents /
find_by / count) to N1QL. Custom/raw JS association views raise
`SimplyCouch::UnsupportedView` (documented boundary).

**Verification:** 20 examples green = 11 N1QL-translator unit tests (no server) +
9 live-Couchbase contract examples (the *same* shared examples couchrest passes).

This is the lower-ambition path: it proves the adapter seam works and gives a
working Couchbase backend, but associations were left as an explicit boundary.

---

## Artifact 3 — `simply_based` (the Couchbase-native ORM)

The path you chose in the thread: not an adapter, a **new gem** that *is* the
ORM, Couchbase-native throughout.

**The one-line swap:**
```ruby
-  include SimplyCouch::Model
+  include SimplyBased::Model
```

**Why so little changes:** simply_couch's model layer (properties, type-casting,
validations, callbacks, dirty tracking, the association DSL, soft-delete,
pagination) is already storage-neutral. SimplyBased keeps all of it and replaces
exactly one layer — `SimplyBased::Store` — which speaks Couchbase directly. Every
association lookup funnels through a handful of Store methods that emit N1QL.

**How the CouchDB views became N1QL:**

- `belongs_to` / `has_many` / `has_one` → scalar foreign-key equality:
  `WHERE ruby_class = $rc AND user_id = $id`, ordered by `created_at`, GSI auto-
  created on first use.
- `has_and_belongs_to_many` → on the storing side, a **KV multi-get** of the id
  array (no query at all); on the other side, the two hand-written map/reduce
  functions become one query:
  ```sql
  SELECT c.*, META(c).id, META(c).cas
  FROM `bucket`.`_default`.`_default` AS c
  WHERE c.ruby_class = $rc
    AND ANY x IN c.`network_ids` SATISFIES x = $v END
  ```
  backed by an array index.
- **Ancestry / trees** → you were right that it's a nested-set/materialized-path
  intention. `path_ids` is the full path root→self, so no recursive build:
  - children = `path_ids[ARRAY_LENGTH-2] = $id`
  - subtree  = `$id IN path_ids` (minus self)
  - roots    = `ARRAY_LENGTH(path_ids) = 1`
- `_rev` is mapped onto **CAS**: a record's `_rev` is the stringified CAS of its
  last write; updates use it for optimistic locking (stale write →
  `SimplyBased::Conflict`).
- Soft-delete-aware `find_by`/`find_all_by`: the `[key, deleted_at] = [val, nil]`
  convention compares element-wise so a `nil` component matches `MISSING`/`NULL`
  (non-deleted docs carry no `deleted_at`).
- Timestamps are microsecond-precision so same-second creations paginate in
  creation order deterministically.

**Replaces Redis too** (same cluster, no second datastore) — 5/5 green:

- `SimplyBased::Counter` — Couchbase binary increment/decrement (true atomic
  counters; a drop-in for Redis INCR/DECR).
- `SimplyBased::Cache` — an `ActiveSupport::Cache::Store` over KV + native TTL;
  usable as `Rails.cache` and therefore the ActiveJob / fragment / low-level cache.
- `SimplyBased::Lock` — distributed lock via insert-if-absent + TTL auto-expiry +
  CAS-checked release (only the holder can unlock).

### Test coverage

SimplyBased runs the **actual simply_couch test suite** against a live Couchbase.

- **403 / 403 applicable examples green on a clean run.** One ancestry reparent
  edge in the namespaced (`by_property`) cascade is intermittently flaky (~50%) —
  it does several dependent saves and occasionally races; tracked, not yet nailed.
- **Green:** CRUD; finders (`find`, `all`, `first`, `find_by_*`, `count_by_*`);
  `belongs_to` / `has_many` / `has_one` / `has_and_belongs_to_many`; validations;
  callbacks; dirty tracking; soft-delete (incl. soft-delete-aware finders);
  pagination; bulk; eager-loading; ancestry/trees; the Redis primitives.
- **Excluded by design:**
  - `:couchdb_only` — native inline attachments, JS-view token-safety,
    design-document maintenance, raw custom JS views. No Couchbase equivalent
    (use `has_s3_attached` / `has_local_attached`; N1QL uses bound parameters).
  - `embedded_in` reverse-lookups — out of scope per your call.

### Known gaps / risks (read before relying on it)

- It is **0.1.0**, written in one session. Treat its youth as the real risk.
- **No multi-document transactions** exposed (Couchbase has them in the SDK; the
  ORM doesn't surface them yet).
- You **design GSIs deliberately** — a query with no supporting index does a full
  primary scan. There is no migration framework for indexes yet.
- `embedded_in` is unimplemented; one flaky namespaced-ancestry test.

### One recommended pattern shift

v0 uses a single collection with a `ruby_class` discriminator (closest to
CouchDB; runs the whole suite as-is). For production, map each model to its own
Couchbase **Collection** — tighter GSIs, cleaner queries, idiomatic. A
`collection :name` per-model DSL is the planned opt-in.

### Integration sketch

```ruby
# Gemfile
gem 'simply_based', git: 'https://github.com/gbentz2/simply_based.git'

# config/initializers/simply_based.rb
SimplyBased.database_url =
  "couchbase://Administrator:password@localhost/app?scope=_default&collection=_default"

# config/environments/*.rb
config.cache_store = SimplyBased::Cache.new(namespace: "app")
```
```ruby
class User
  include SimplyBased::Model
  property :name
  has_many :posts
  validates :name, presence: true
end
```

---

## Should this ORM exist? — SimplyBased vs. Postgres + Redis

You raised the real question: another path is to drop CouchDB and refactor to a
standard Rails stack (Postgres + Redis). For an app with **many reads and writes
per second**, does a Couchbase-native ORM have real benefits? An honest take.

**Maturity first.** Postgres + ActiveRecord + Redis is the most battle-tested
stack in Rails. SimplyBased is 0.1.0 with known gaps. On risk and ecosystem
alone, the standard stack wins decisively. That sits at the top of the ledger.

| Dimension (sustained high throughput) | SimplyBased / Couchbase | Postgres + Redis |
|---|---|---|
| Raw write scale | memory-first, async persist, sub-ms KV, **shards horizontally** | single primary takes all writes; scales up, not out (shard/partition = heavy) |
| By-id reads | sub-ms from RAM | fast, network + planner hop; replicas for read scale |
| Complex/ad-hoc queries | N1QL, but **you design every GSI**; missing index = full scan | world-class planner, joins, ad-hoc SQL |
| Transactions / integrity | single-doc atomic + CAS only (multi-doc not exposed) | **ACID, FKs, constraints, multi-row txns** |
| Operational surface | **one datastore** — docs + cache + counters + locks, no Redis | two systems, two backups, PG↔Redis cache-invalidation drift |
| Cost / footprint | RAM-hungry, multi-service, doesn't scale down; some EE-licensed | both lean, cheap, scale down; fully open |
| Schema | schemaless documents, denormalized aggregates | migrations — but **`jsonb` gives documents *with* ACID** |

**Verdict.** Couchbase's write advantage is real but only **above a high ceiling**
— a tuned Postgres does tens of thousands of writes/sec; you only *need*
Couchbase's horizontal write scale-out when you'd otherwise be sharding Postgres
(the genuinely painful threshold). Below it, Postgres + Redis wins on
transactions, cost, ecosystem and hiring.

So it's not "which is faster" — it's **data shape and scale horizon**:

- **SimplyBased/Couchbase wins** when the domain is document-shaped and
  denormalized, cross-entity transactions are rare, you truly expect to outgrow a
  single Postgres primary, and the **one-datastore consolidation** (no Redis, no
  cache-invalidation drift) is worth running a heavier engine.
- **Postgres + Redis wins** for relational/transactional domains (most business
  apps), the mature default Rails path, cost-sensitivity, or when `jsonb` already
  covers your document needs.

**Migration-cost asymmetry — and it favors SimplyBased for *this* project.** You
are already on CouchDB with simply_couch models:

- *SimplyBased path:* `include` swap, design GSIs, adopt the collection pattern,
  copy data CouchDB→Couchbase. Document model + association API survive intact.
  Low app-code churn.
- *Postgres path:* re-model documents into tables (or jsonb), rewrite every model
  to ActiveRecord, rebuild associations as FKs, re-express views as SQL, rewire
  jobs/cache. You land on the best-supported stack — but it's a large rewrite.

**The option neither of us named first, and often the strongest:**
**Postgres + `jsonb` + Redis.** Document flexibility *and* ACID, on the mature
Rails rails, with Redis still doing cache/counters/locks. For a transactional
high-throughput app this is the "boring tech wins" answer.

**Bottom line.** SimplyBased earns its place when the document model is the point,
write scale-out is a real future need, and consolidating onto one engine is worth
the heavier footprint and the youth of the library. For getting *this* project off
CouchDB with minimal model-code churn, it's the lowest-friction path and it's
genuinely working (403/403 applicable). But the benchmark gap only pays off past a
scale most apps never reach — so weigh the 0.1.0 maturity and the
multi-doc-transaction gap as the real decision factors, not the throughput
numbers.

---

## Repos & access

| Artifact | Location | State |
|---|---|---|
| Adapter refactor | PR https://github.com/bterkuile/simply_couch/pull/10 | open |
| Couchbase adapter gem | https://github.com/gbentz2/simply_couch-couchbase | public |
| SimplyBased ORM | https://github.com/gbentz2/simply_based | **private** |

To browse `simply_based`, the cleanest options are: Greg flips it public, adds
your `bterkuile` GitHub account as a collaborator, or mints a read-only
fine-grained deploy token scoped to that one repo. (A link with Greg's personal
token embedded works for `git clone` but hands over his whole account — not
recommended for anything durable.)

## Reproduce the verification

- Ruby 3.1+ (built on 4.0.5). `bundle install`.
- CouchDB: `docker run -d -p 5984:5984 -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=secret couchdb:3`
- Couchbase: `docker run -d -p 8091-8096:8091-8096 -p 11210:11210 couchbase:community`,
  then init a single-node cluster + a **couchstore** bucket (Community edition
  rejects the default magma backend) named `simply_based_test`.
- `COUCHBASE_URL="couchbase://Administrator:password@localhost/simply_based_test" bundle exec rspec`