#3240 Datastore rework Phase 3: the commit-log unit of work and binding registry (no v2 behaviour change)

Background (read this first)

swamp is reworking its datastore layer (tracking: swamp-club#2865). In the target design, v3, every change is an explicit commit in a single-writer SQLite commits.db (replicated by Litestream), and content is stored as content-addressed chunks. Phase 3 builds the v3 engine dormant: nothing reaches it until a user opts a new, empty datastore into v3, which is a later issue.

How writes are routed today (main at 1b48d5b4), from Phases 1 and 2:

• Staging. Repositories stage every change through signalChange(markDirty, change) (src/infrastructure/persistence/unit_of_work_scope.ts). StagedChange is write | remove (a path) or bulk (a reason). It has three routes:
  1. if the ambient unit of work is bound to the repository's own hook (legacyUnitOfWorkTarget(uow) === markDirty), stage into it;
  2. otherwise call the hook directly and warn once per call site. That's route 2: it still exists, and its removal is deferred;
  3. with no hook (filesystem datastores), do nothing.
• Legacy units. createLegacyUnitOfWork (legacy_unit_of_work.ts) forwards each staged change to the hook at once. runInRootUnitOfWork (repo_unit_of_work.ts) opens one root per CLI command or serve request: it flushes (pushes) once at the end, and checkpoint() pushes partway through. Use cases open child units that roll up into the root.
• The behaviour suites pin all of this:
  • integration/usecase_sync_characterization_{access,data,extension,model_workflow,vault}_test.ts;
  • usecase_sync_root_unit_harness_test.ts, serve_root_unit_test.ts and cli_root_unit_failure_test.ts;
  • repository_dirty_coverage_test.ts and datastore_peer_propagation_test.ts;
  • the unscoped_write_guard;
  • the fitness rules in datastore_write_seams_rules_test.ts.

What's on main for v3, all dormant. The design is in design/enablers/datastores.md under "v3 domain model", "v3 commit store" and "v3 chunk storage":

• The domain model (swamp-club#3233, src/domain/datastore_v3/): PartitionKey, Intent, the ops, Commit and Partition. commit_log_unit_of_work.ts already proposes the ports this issue implements:

  • TypedChange { partition, op };
  • CommitLogUnitOfWork extends UnitOfWork, adding stageOp, stagedOps and checkpoint;
  • UnitOfWorkBinding, with bind and boundHook.

  The domain-model section also records the decisions this issue implements:

  • (1) typed changes come through the sibling port;
  • (2) a binding registry generalises the hook-identity check;
  • (3) writes outside a unit are refused on v3, including a hooked write inside the CLI's managed-lockfile root (swamp-club#3192), whose root is bound to the transaction's own hook;
  • (4) abandon discards staged ops;
  • (5) a run's unit commits on completed, failed, suspended and cancelled, and discards on an error or an early stop;
  • (6) checkpoint commits the partitions touched so far and keeps the unit open;
  • (7) data partitions commit first and a run record last, with no atomicity across partitions.

• The commit store (swamp-club#3236): CommitStore.append(commit) returns appended or duplicate, and refuses with CommitRejectedError.

• The chunk store (swamp-club#3237): ChunkStore.write(namespace, content, signal) returns a BlobRef. Chunks are uploaded before any commit names them.

A decision this issue builds on: on v3 in Phase 3, repositories keep writing local files as today. The read model, a parallel issue, applies commits from the store, and applying one twice changes nothing. So this issue never calls the read model, and the read model never calls into this one.

Goal

Implement the commit-log unit of work and the binding registry, so that a unit bound to a v3 repository context commits typed changes to commits.db. Writes outside a unit are refused on v3. v2 behaviour doesn't change at all. Repositories don't stage typed ops yet: that comes next, in its own issues.

Design

1. The binding registry. Implement UnitOfWorkBinding, and change signalChange to ask it instead of legacyUnitOfWorkTarget. The legacy adapter becomes one registrant, binding each unit it builds to its hook exactly as today. That covers roots, children, nested roots over another hook, and units built with a wrapped hook, which must still not match. Every v2 routing decision must come out identical. Keep legacyUnitOfWorkTarget's behaviour available, or delete it only if nothing else needs it.
2. CommitLogUnitOfWork, in infrastructure (e.g. src/infrastructure/datastore_v3/commit_log_unit_of_work.ts):

   • Staging. stageOp(typedChange) records typed changes in order. stage(StagedChange), the path-only form, is refused on v3 with a clear error naming the change: there's no path-to-op guessing.

   • commit():

     • groups ops by partition;
     • uploads content that needs a chunk first, through ChunkStore.write, so that no commit names a missing chunk;
     • builds one Commit per partition: seq from CommitStore.head plus one, a fresh txnId per commit, the writer identity, and the fence from the writer's lease or lock when one is given;
     • appends in decision 7's order.

     Retrying the same txnId after a failure is a no-op through the store's duplicate rule. A refused commit surfaces CommitRejectedError unchanged. Document what happens to partitions already committed in the same unit: no atomicity across partitions.

   • abandon() discards staged ops (decision 4). checkpoint() commits the partitions touched so far and keeps the unit open (decision 6).

   • Children and run outcomes. Child units roll up into their root, as legacy units do. Wire the run-outcome rules (decision 5) the way the legacy units handle pushWhen.

   • No outbox yet. In Phase 3 the unit appends straight to the local CommitStore. The outbox is Phase 4: leave a seam for it.

3. Unscoped writes on v3. With a v3 binding in place, signalChange must refuse a write that has no bound unit, rather than take route 2. On v2, route 2 behaves exactly as today. Add a test for the #3192 case: a hooked repository write inside a CLI lockfile transaction on v3 is refused.
4. Factory. Provide a way to open a commit-log root over a v3 repository context, mirroring runInRootUnitOfWork: commitStore, chunkStore, namespace, writer identity, optional fence. It is used only by tests until the opt-in.
5. Dormancy. Extend integration/datastore_v3_rules_test.ts so no production code outside the v3 directories imports the commit-log unit. The binding registry is the one shared production change; pin it in datastore_write_seams_rules_test.ts the way signalChange and createLegacyUnitOfWork are pinned.

The gate: no behaviour change on v2

• Record first. Before changing signalChange, add rows to the characterization suites for the routing cases: a root, a child, a nested root over another hook, a wrapped hook, and the #3192 lockfile root. Commit them on their own. Show in the PR that they pass on that commit.
• Then change nothing in them. Every suite listed in the background must pass with no expectation edits after the registry lands.
• Prove it on every route. Run the whole suite with the route-2 recorder approach used for #3056 and the route-2 research: a temporary patch in signalChange that logs every write made with no bound unit, or with a unit bound to another hook. Show that the recorded set is identical before and after the change.

Tests

• Unit tests for the registry: bind, look up, wrapped hooks not matching, nested roots over two hooks.
• Integration tests on a real temp directory, with the SQLite commit store and the filesystem storage/1:
  • staging and commit: typed changes staged and committed land as the expected commits;
  • ordering: data partitions land before the run record;
  • content: multi-chunk content is uploaded before its commit;
  • retries: a retry after an injected failure lands once;
  • refusals: a refusal surfaces unchanged;
  • unit lifecycle: abandon discards, checkpoint commits and keeps the unit open, children roll up, and each run outcome follows decision 5;
  • unscoped writes: an unscoped hooked write on v3 is refused, and the #3192 case is refused.
• The v2 gate above.

Out of scope

• Repositories staging typed ops. That's the next issues, split into three groups: data and output; definition, workflow and evaluated; run and vault config.
• The read model (a parallel issue), the outbox, Litestream, serve and the opt-in.
• Removing route 2 on v2, which is deferred.

Done when

• The registry is live with identical v2 behaviour, proven by the gate.
• The commit-log unit and its factory are on main, dormant, with these tests.
• datastores.md records the registry, the commit order and failure semantics, and the outbox seam.

Blocked by: nothing (#3236 and #3237 have merged). It runs in parallel with the v3 read-model issue.