#3491 Datastore rework Phase 4: close D2 follow-ups before v3 replication is switched on (lineage, messages, migration, chunk GC)

Background (read this first)

swamp is reworking its datastore layer (tracking: swamp-club#2865). In the new format, v3, every change is a commit in a single-writer SQLite commits.db. Content is stored as content-addressed chunks, and each checkout rebuilds its files from the commits. v3 is opt-in today for filesystem datastores (swamp-club#3279), and swamp-uat's filesystem-v3 backend tests it in CI.

Phase 4 is "solo mode on buckets":

• the CLI is the single writer of a local commits.db;
• Litestream replicates it to the bucket;
• chunks go to the bucket through storage/1;
• a fenced writer lease makes sure only one machine writes.

Shipped so far:

• A, swamp-club#3357: replication cost, choosing an idle sidecar.
• B, swamp-club#3358: the pinned, hash-checked Litestream binary.
• C, swamp-club#3359: the fenced writer lease (src/domain/datastore_v3/writer_lease.ts, src/infrastructure/datastore_v3/v3_writer_lock.ts). Every CLI write command takes a new fence, serve keeps one for its lifetime, and commits.db keeps a store-wide writer_fence that refuses lower fences.
• D1, swamp-club#3416: a dormant Litestream sidecar (src/infrastructure/datastore_v3/litestream/, src/domain/datastore_v3/litestream_replication.ts and replication_lineage.ts).
  • A write command confirms its commits are replicated (POST /sync with wait, replicated_txid >= txid, then the lease check) before it succeeds.
  • Replication follows a lineage: the replica path is _litestream/<fence>/, named by the fence at which this copy's replication began. Another writer's fence restarts it on a new path. The lineage is recorded in lineage.json beside commits.db, with abandoned ones in previousLineages.
  • Nothing replicates unless the internal SWAMP_V3_REPLICA setting (replica_setting.ts, tests only) names a file or S3 replica.
• D2, swamp-club#3482: restore, refresh and takeover, also dormant.
  • While replicating, each checkout keeps its own commits.db under .swamp/_datastore_v3_copies/.
  • Lineages are published under _control/lineage/<fence>/ (started.json, head-<writer fence>.json, confirmed.json).
  • A restore truncates to the published head's high-water mark.
  • See "Phase 4D2" in datastore-litestream.md.

The decisions are in design/enablers/datastore-litestream.md ("Decision", "Restore, failover and fences (Q6)", "Phase 4D1") and in design/enablers/datastores.md (the "v3 …" sections).

Goal

Close the follow-ups D2's own verify reviews left for before Phase 4E switches replication on (PR #3131, "Before 4E switches replication on"). None of them matter while replication is dormant. All of them do once it's live.

The items

1. prepareV3WriterLineage during serve's retake: concurrent calls can fail one request with a misleading "already claimed". Serialise it per copy.

2. Fenced message wording:

   • in serve's retake window, the message says another process took the lease, although serve holds it again;
   • the "published" fenced message is also used when the lease check fails for another reason (an I/O error).

   Make each message match what happened.

3. copy.json can move back to an older lineage. Guard it by lineage fence so it only moves forward, and fix the comment that says it never moves backwards.

4. prepareV3WriterLineage's conflict path logs "a new lineage starts" but reuses the old lineage, and a same-fence claim under another replica root is accepted. Make the log true, and refuse a same-fence claim under another root (or say why it's safe).

5. Two doc comments say a missing copy is "renamed into place"; it is linked.

6. swamp datastore namespace list reads the checkout's copy without refreshing it. Refresh it as other reads do, or say in its output that it may be behind.

7. Migration. Turning replication on over a datastore that already has a shared commits.db starts every checkout from an empty copy and silently ignores the shared commits. Publish the shared database as the first lineage when replication is first turned on for a datastore, under the lease, once. Test it: an existing datastore's commits are all present in a fresh checkout's restored copy afterwards.

8. Chunk GC and truncation. A restore's truncation releases chunk references that another copy may still restore. Make sure chunk GC can't delete a chunk a restorable lineage head still names, and test it. The record flags this for 4E's chunk GC.

Tests

• One test per item, each failing without its fix where the item is a behaviour.
• The D1 and D2 suites pass unedited.
• With SWAMP_V3_REPLICA unset, nothing changes.

Blocked by: nothing. Phase 4E waits for it. It can run in parallel with the serve confirmation issue and F; whichever lands second rebases.