#3234 Datastore rework Phase 3: Litestream embedding spike (a written decision, no production code)

Background (read this first)

swamp is reworking its datastore layer (tracking: swamp-club#2865). In the target design, v3, every change is an explicit commit in a single-writer SQLite database, commits.db. Litestream continuously replicates that database to object storage (S3, GCS and others), so it can be restored anywhere:

• Solo mode (Phase 4). No serve is configured. The CLI's embedded datastore service is the single writer: it keeps commits.db on the user's machine, content chunks go to the bucket, and Litestream ships the database's changes to _litestream/ in the same bucket. A solo writer lock lives at _control/writer.
• Team mode (Phase 5). One elected swamp serve leader holds commits.db on local disk and replicates it with Litestream. On failover, a follower takes the leader lease (with a higher fence) and restores commits.db from the replica. The plan is a Litestream generation per fence.
• Durability promise. The CLI keeps each commit in a local outbox until the writer confirms Litestream has replicated it. So the design needs a way to know when a given transaction is replicated.
• Platforms. swamp ships as a single Deno-compiled binary for Linux, macOS and Windows (amd64 and arm64). It already uses SQLite for _catalog.db; find out which binding and how it's built. Windows is supported but not first-class: Windows CI runs only on pushes to main.

The public design write-up is the "Swamp Datastore Redesign" page. Phase 4's "embedded datastore service in the CLI" issue can't start until this spike has produced its decision.

Goal

Decide how swamp runs Litestream, and write the decision down. This is a spike: the output is a written decision with evidence, not production code. Throwaway prototypes go in a scratch branch, not on main.

Questions to answer, with evidence

1. Embedding model. Compare these options on binary size, startup cost, licensing (Litestream is Apache-2.0), upgrade path, crash behaviour and how swamp would ship it:
   • Litestream's Go library linked into a helper;
   • the litestream binary run and supervised as a sidecar by swamp;
   • a Deno/TypeScript reimplementation of the WAL-shipping parts.
   • Anything else credible, for example LiteFS, or SQLite's own session or WAL APIs from Deno.
2. Compatibility with swamp's SQLite. Litestream needs WAL mode and control of checkpoints. Litestream must be the only process checkpointing, or the app must cooperate. Check this against the SQLite binding swamp uses (Deno's node:sqlite, jsr:@db/sqlite or whatever _catalog.db uses) and against how a compiled deno compile binary can load or ship extra native code.
3. Object storage targets. Can it replicate to S3, GCS and a local directory (the filesystem backend), using the credentials the S3 and GCS datastore extensions already use? Or would swamp need its own replica client? Also: how does the replica sit under a bucket prefix and a swamp namespace?
4. Knowing a commit is replicated. How does the writer learn that the WAL frames containing a given transaction have reached the replica? That is what the outbox waits for. What's the typical replication lag, and can a sync be forced?
5. Shutdown and crash. Does a final sync on a clean exit (CLI command end, serve shutdown drain within --shutdown-drain-timeout, default 30 s) happen reliably, and what does it cost on short CLI commands? After a hard kill, what's lost, and does the outbox cover it?
6. Restore. Find the time and correctness of restoring commits.db on a fresh machine and on a failover node. Find out how generations work, and whether a generation per leader fence fits.
7. Windows and macOS. Does the chosen model work on Windows (amd64 and arm64) and macOS? Cover file locking, process supervision and path handling. If Windows can't be supported, what's the fallback? Options include a different replication path, or solo mode only on Windows with a warning.
8. Short-lived processes. Most CLI commands live for seconds. Should each command start replication and sync at its end, or should a longer-lived process own it? Measure the per-command overhead.

Deliverable

A decision record in design/enablers/ (for example datastore-litestream.md), linked from design/enablers/datastores.md, containing:

• the chosen embedding model and why, with the alternatives and why not;
• measurements: binary size delta, startup cost, per-command sync cost, replication lag, restore time;
• the Windows and macOS answer;
• how "replicated" is detected for the outbox;
• the shutdown and crash behaviour;
• how the replica is laid out in the bucket (prefix, namespace, generations);
• the risks, and anything that changes the Phase 4 or Phase 5 plan.

Put the prototype's branch name and the commands you used to measure in the record, so the numbers can be reproduced.

Out of scope

Production code, commits.db's schema (the commit-store issue), the outbox, leader election, and changes to the S3 and GCS extensions.

Done when

The decision record is merged, and it answers all eight questions with evidence, or states plainly what couldn't be answered and what would answer it.

Blocked by: nothing. It runs in parallel with the Phase 3 domain-model issue.