#3192 Datastore rework Phase 3: publish the extension lockfile as a unit-of-work commit (no behaviour change)

Background (read this first)

swamp is reworking its datastore layer (tracking: swamp-club#2865; design: design/enablers/datastore-commit-log.md). The aim is that every change a command makes is staged into a unit of work, which pushes it to the datastore. Commands should never mark files dirty or call push themselves.

Phases 1 and 2 are done (main at a7ab3774). The pieces this issue builds on:

• The port. UnitOfWork (stage, commit, abandon, staged) with a legacy adapter that forwards each staged change to the MarkDirtyHook (#2970). Repositories stage through signalChange into the ambient unit (#2971).
• Root units. Every command and serve request runs in one: runInRootUnitOfWork in src/infrastructure/persistence/repo_unit_of_work.ts. The root runs its flush once when it ends, on every outcome, with pushWhen: "always" | "completed" (#3032, #3033, #3034, #3055). Use-case units are children that roll up into the root.
• Checkpoints. root.checkpoint() pushes partway through an operation, for code that must publish before it carries on (#3053). It returns Promise<void> today.
• One flush path. Pushes go through src/infrastructure/persistence/push_paths.ts (#3055). PINNED_DIRECT_PUSHES in integration/datastore_write_seams_rules_test.ts lists every production push call that isn't a root's flush or checkpoint, each with a reason.

The lockfile is the last hand-marked, hand-pushed writer. It was deferred from Phase 1 and Phase 2 because its mark isn't a record of a write: it is a publish signal. That is the job of this issue.

How the lockfile works today

The managed extension lockfile is shared between checkouts through the datastore (swamp-club#2838). ManagedLockfileTransaction in src/libswamp/extensions/managed_lockfile_transaction.ts wraps every lockfile change. Under the datastore global lock it:

1. reads the pending record from .swamp/ (createRepoPendingLockfileStore). This is a change an earlier process failed to publish;
2. fetches the datastore's lockfile (hydrate, a pullChanged of the config tier);
3. replays the pending change onto it with LockfileRepository.replaceAll;
4. runs the caller's change, which writes through LockfileRepository (writeEntry, removeEntry, restoreEntries). Those writes don't mark anything;
5. in #settle, records the whole outstanding change as pending, then publishes: createDatastoreLockfileSync.publish calls markDirty(lockfilePath) and then pushChanged directly, bounded by runBoundedSync;
6. clears the pending record once the publish succeeds.

Two details make this a publish signal rather than a write:

• mustUpload. When the lockfile differs from the fetched copy, a push that reports 0 files sent is a failure (managed_config_unpublished). This guards against a sync service that lost track of its dirty files.
• The pending retry (earlierChangeOnly). A later process whose own change did nothing still marks and pushes the lockfile, to publish an earlier failed change. So it pushes in a process that wrote nothing.

On failure:

• CLI: publishFailure: "throw". The change is recorded and ManagedLockfileUnpublishedError is raised once per-item results are reported.
• Serve: "defer". It warns, the request succeeds, and the next transaction retries.
• Either way, a failed caller change still settles, and the original error is rethrown.

Wiring:

• CLI: buildManagedLockfileTransaction / createManagedLockfileTransaction in src/cli/managed_config_sync.ts, used by extension install, pull, rm, search, update, doctor extensions and repo init.
• Serve: extensionLockfileTransaction in src/serve/handlers/admin_handlers.ts, taken inside the handler's exclusive sync gate.

The pins this issue should remove, all in integration/datastore_write_seams_rules_test.ts:

• the hand-mark pins createDatastoreLockfileSync and admin_handlers.ts: extensionLockfileTransaction;
• the PINNED_DIRECT_PUSHES entry for createDatastoreLockfileSync.

The PINNED_REPO_CONSTRUCTIONS entry ManagedLockfileTransaction: LockfileRepository can stay unless your design removes it.

Goal

The lockfile publish becomes a unit-of-work commit:

• the lockfile change is staged into the command's or request's root unit;
• it is published by the root's checkpoint, while the global lock is still held;
• nothing marks or pushes by hand.

Behaviour stays exactly as it is today: the same pushes at the same points, the same mustUpload failure, the same pending record and retry, the same CLI throw and serve defer, and the same errors and codes.

Design

1. Record behaviour first, in a separate commit before any conversion. Extend the characterization suites (integration/usecase_sync_characterization_*) and the managed-lockfile tests with rows for:

   • a CLI install, rm and update that change the lockfile: marks, pushes, order relative to lock release, and the pending record after;
   • a push that sends 0 files while mustUpload is set: error, code, and the pending record kept;
   • a pending record from an earlier process plus a command whose own change is empty: it still marks and pushes the lockfile, then clears the record;
   • the pending unknown kind (an older swamp's record): the warning and the merge;
   • a caller change that throws after writing: it settles, publishes, and rethrows the original error;
   • serve defer: the publish fails, the request succeeds, the warning appears and the record is kept;
   • a failed publish followed by a failing release.

   These expectations must not change in later commits. Show in the PR that they pass on the commit before the conversion.

2. Stage, don't mark. The transaction's publish stages one write change for the lockfile path into the ambient root unit, in place of markDirty. This includes the pending-retry case, where the process wrote nothing else.

   Don't make LockfileRepository stage on every write. It also writes lockfiles outside any managed transaction (filesystem datastores, repos without managed config), and staging there would add marks that don't happen today. If your design needs repository-level staging, show with the characterization rows that no new marks or pushes appear.

3. Publish with a checkpoint. The publish is the root's checkpoint(), called inside the transaction before the global lock is released, through the push paths in push_paths.ts.

   • mustUpload. It needs the pushed count, so let the checkpoint, or the push path behind it, report how many files it pushed. Keep the "0 files sent" check and error exactly as they are.
   • The time limit. Keep runBoundedSync's bound and labels.

4. Roots with no checkpoint. Every CLI and serve caller must run in a root that has a checkpoint. A transaction with no ambient root, or with a root without a checkpoint, should fail loudly in tests, not fall back to a hand push. Use the route-2 test seam (useUnscopedChangeReporterForTesting) pattern if it helps.

5. Serve. extensionLockfileTransaction gets the same staging and checkpoint. The order stays the same: sync gate, then the global lock, then the publish, then the release. defer stays.

6. Remove the three pins listed above. Update design/enablers/datastores.md to say the lockfile publishes through the root's checkpoint.

Tests

• Characterization rows from design step 1, recorded before the conversion and unchanged after it.
• Unit tests for the transaction against a fake root:
  • the stage happens before the checkpoint;
  • the checkpoint runs before the lock is released;
  • a 0-count push with mustUpload fails;
  • an empty change with a pending record still stages and checkpoints.
• The fitness rules pass with the three pins removed.
• Repeats. The managed-lockfile and characterization tests pass under --repeats=3.

Out of scope

• The format-guard cost work (swamp-club#3191). It runs in parallel and touches different files.
• Removing signalChange's route-2 hook fallback. It is deferred and tracked on #2865.
• The v3 commit log. On v3 the lockfile will be a commit like any other.

Done when

• No hand mark or hand push of the lockfile remains, and the three pins are gone.
• Every recorded row passes unchanged on the old and the new code.
• The PR lists every lockfile caller, CLI and serve, and the root and checkpoint each runs under.

Blocked by: nothing. The one-flush-path issue (#3055) and checkpoints (#3053) have merged.