#2838 managedConfig: extension writes update the shared lockfile from a stale cache and overwrite peers' entries

Summary

Split from swamp-club#2495 (part a). In a managedConfig repo on an extension-backed datastore (S3/GCS), extension write commands read, modify and write the cache copy of the shared lockfile (config/upstream_extensions.json) without first fetching it from the datastore, and then publish that file. The S3/GCS push is last-writer-wins per file, so a write from a stale cache overwrites entries a peer added after this checkout last pulled:

• On a checkout that has not pulled the config tier since a peer's write, swamp extension pull X publishes a lockfile without the peer's entries.
• On a stale cache, swamp extension rm reverts additions made by peers.

State on main (fe52cda6)

Fixed since #2495 was filed:

• Publishing pushes only the changed path. Extension writes use pushManagedConfigPathsDeferred (src/cli/managed_config_sync.ts:179, swamp-club#2429).
• A failed publish throws ManagedConfigUnpublishedError instead of only warning.
• A per-checkout pulled-extensions lock, with LockfileRepository.refresh() under it (swamp-club#2709): src/libswamp/extensions/pull.ts:899 and src/libswamp/extensions/remove_extension_service.ts:168.

Still broken:

• extension pull, update, rm and search install call requireRepoMarker, which does not pull, and then resolveManagedLockfileForWrite (src/cli/repo_context.ts:492), which resolves the lockfile path but never fetches the file. refresh() only re-reads the local cache file.
• createExtensionInstallDeps (src/cli/create_extension_install_deps.ts:66), used by extension install, repo init and serve, has the same shape.
• Serve's extension handlers (src/serve/handlers/admin_handlers.ts ~705-760) write the lockfile and then call pushChangedToRemote, also without fetching first. The 30 s config poller narrows the window but does not close it.
• upstream_extensions.json.lock (src/infrastructure/persistence/lockfile_repository.ts:209) sits next to the lockfile in the synced directory. The extensions' isInternalCacheFile excludes only a bare .lock, so it can be uploaded.

Design (decided on #2495)

• Stage the download, verification and extraction outside any lock.
• Then, under the datastore global lock: fetch the shared lockfile into the cache, write the entry, mark that path dirty, run the scoped push, and release the lock.
• Fetch with pullChanged scoped to config with the namespace, or with a non-persisting fetch followed by an atomic local write. Do not use S3 hydrateFile → pullFile: it never binds the namespace (a bound serve singleton writes to <cache>/<ns>/<ns>/…), it is not atomic, and it does not update the index.
• The fetch must happen under the lock. An unlocked scoped pullChanged slow path re-downloads any file that differs, which reverts other processes' unpushed writes.
• The advisory lockfile lock (10 × 100 ms) cannot be held across network I/O, so use the datastore global lock.
• Lock order: datastore global lock → pulled-extensions lock → the lockfile's own .lock. Take the global lock before the pulled-extensions lock, never inside apply: the auto-resolver runs inside commands that already hold the global lock.
• Move the transaction lock out of the synced directory, or make sure it is never uploaded.
• Filesystem datastores skip all of this.

Out of scope (other parts of #2495)

• (b) The auto-resolver writing and publishing the shared lockfile, including serve's auto-resolver port and gate re-entrancy.
• (c) Adopting extension files already on disk.
• (d) Retiring the legacy in-repo lockfile.
• (e) datastore setup extension overwriting the config tier (swamp-club#2837).

Tests

• A ManagedLockfileSyncPort { hydrate, publish } seam with a temp-dir fake remote.
• extension pull from a stale cache keeps a peer's entry, and extension rm from a stale cache does not revert a peer's addition.
• A failed publish followed by a fetch.
• The .lock file is never uploaded.
• Manual end-to-end: two instances sharing a MinIO bucket.

Related

• swamp-club#2495: parent issue (this is part a)
• swamp-club#2429, swamp-club#2709: prerequisites (shipped)
• swamp-club#2612: CLI convergence (unit I) depends on this