#2495 managedConfig: extension lockfile writes lose updates, and auto-resolved installs never reach the shared lockfile

Description

Split from swamp-club#2483 during triage on 2026-09-24. #2483 fixes how the managed config base is resolved at startup, and how extension list reads the managed lockfile. This issue fixes how the managed extension lockfile is written. It is related to swamp-club#2429 and should land before it.

Three problems exist today in managedConfig repos on extension-backed datastores (S3/GCS).

1. Lost updates to the shared lockfile

• extension pull, update and rm use requireRepoMarker, which does not pull. They read-modify-write the cache copy of config/upstream_extensions.json.
• They then push with pushManagedConfigChangesDeferred. That push is also done without a pull (managed_config_sync.ts ~62-80), and the S3 push is last-writer-wins per file.
• On a fresh machine, the first swamp extension pull X writes a lockfile that contains only X, and that overwrites the team's remote lockfile.
• On a stale cache, rm reverts additions made by peers.

2. Auto-resolved installs never reach the shared lockfile

• The auto-resolver records installs in the instance-local in-repo .swamp/config/upstream_extensions.json.
• #2483 keeps this for now. Its loaders read that file read-only alongside the shared lockfile, as a transitional measure, so auto-resolved types stay visible.
• Auto-resolved sets therefore differ per instance.
• Pins and dependencies are read from the local file only.

3. The "files already exist" dead end

• When extension files sit on disk but the lockfile has no entry, the auto-resolver installs with force: false, hits ConflictError and gives up (auto_resolver_adapters.ts ~236-262).
• That leaves the type unresolvable until someone runs swamp extension pull X --force.

Target design (decided on #2483)

• A managed repo has a single shared managed lockfile at the resolved config base.
• Every writer hydrates it from the datastore before writing, and publishes it after. The writers are: explicit extension commands, the auto-resolver, doctor --repair, search install, and the repo upgrade install pass.
• The auto-resolver writes and publishes the shared lockfile. Auto-resolved extensions become team-wide.
• The auto-resolver adopts extension files that are already on disk instead of dead-ending. It verifies them against the legacy entry's filesChecksum or a checksum-verified registry archive, and never touches the files. The swamp-club#121 local-edit guard is kept.
• The legacy in-repo lockfile is merged once and retired. This removes #2483's transitional read.

Constraints found by the #2483 adversarial reviews

The full findings are recorded on the #2483 lifecycle as ADV-93, ADV-119 to ADV-130 and ADV-132.

Legacy path can be the live lockfile (critical).

• <repo>/.swamp/config/upstream_extensions.json is the live managed lockfile for a filesystem datastore at its default path ({repo}/.swamp), and whenever config is not a datastore subdir.
• Merge and retire only when the realpaths differ, and only for extension-backed datastores.

Hydrate primitive.

• S3 hydrateFile → pullFile never binds the namespace. The correct relPath depends on the instance's history, and a bound serve singleton writes to <cache>/<ns>/<ns>/….
• pullFile is not atomic, and does not update the index.
• Use pullChanged scoped to config with the namespace, or a non-persisting fetch followed by an atomic local write.
• The two bugs above should be filed against swamp-extensions (the S3 and GCS cache sync).

Locking.

• An unlocked hydrate reverts other processes' unpushed writes: a scoped pullChanged slow path re-downloads any file that differs.
• The advisory lock (10×100ms) cannot be held across network I/O.
• Stage download, verify and extract outside any lock. Then, under the datastore global lock: hydrate, write the entry, markDirty that path, run a scoped push, and release.
• Filesystem datastores skip all of this.

Publish.

• pushManagedConfigChanges calls a bare markDirty(), which pushes the whole cache, and does it without the global lock, on a second sync instance.
• A failed publish only warns, so the next hydrate drops the entry.
• upstream_extensions.json.lock sits in the synced dir and can be uploaded; isInternalCacheFile only excludes a bare .lock.
• Publish must be path-scoped. A failed publish must fail the command, or persist a marker that makes the next hydrate merge. Move the transaction lock out of the cache.

Held locks (architecture).

• The auto-resolver runs inside CLI commands that hold the global lock (model validate/evaluate, workflow evaluate).
• In serve, it runs inside exclusively gated handlers (model.create, vault.create, vault.migrate), and ungated inside runs. The sync gate is not re-entrant.
• Needed: a coordinator-level sync-session accessor. Inside a locked command, write, markDirty the path, and let the flush publish.
• Serve needs its own auto-resolver port, bound to its singleton sync service and gate: a re-entrant gate, or a post-unit publish queue. Update SYNC_GATED_REQUESTS, UNGATED_PUSH_HANDLERS and integration/serve_deps_rules_test.ts.

Recovery pull and datastore bootstrap.

• Extension commands must resolve installed-only (#2483 adds that option).
• Install into a scratch LockfileRepository, then replay the entries into the hydrated shared lockfile after resolution.
• Deferred records are flushed on the unresolved-to-resolved transition, and never overwrite an existing shared pin (swamp-club#465). Warn about the version skew instead.
• Make any exemption content-based: the staged archive's datastores/ provides the marker's type. Limit it to extension pull and the auto-resolver's datastore path.

Adoption.

• Needs a new InstallationInspection state (unrecorded), an adopt() port, and an output event and renderer.
• Refactor extraction into a shared stageExtension/commitStaged: the raw archive layout differs from the installed layout (manifest header, bundles and skills live outside the root).
• Fix pull computing filesChecksum before orphan pruning (pull.ts ~1177 versus ~1187).
• Adoption reaches only trusted collectives.

Legacy merge on a name clash.

• Prune the union of both entries' file lists.
• Record the on-disk datastore extension version when it matches the legacy entry.
• Retire to a timestamped .migrated.

Serve install.

• createExtensionInstallDeps should take lockfilePath plus a sync port.
• Serve passes resolveManagedPathsFromContext, and ctx.syncService without re-gating.

Tests.

• A ManagedLockfileSyncPort {hydrate, publish} seam with a temp-dir fake remote.
• A legacy path equal to the live path.
• A JSON-mode exempt pull with the auto-resolver configured.
• A pin not overwritten by a deferred record.
• The .lock file never uploaded.
• A failed publish followed by a hydrate.
• Two instances sharing a MinIO bucket (manual e2e).

Docs.

• design/enablers/datastores.md: the "where mutations write" table (auto-resolve row, hydrate step); serve's auto-resolver lockfile write; gate re-entrancy.
• The manual's Known limitations entry on last-writer-wins.
• The swamp skill's repo/references/structure.md ~124-128 and share/references/joiner-instructions.md ~80-84 ("the repo's lockfile pins"). The swamp skill is bundled, so tessl must stay at or above 90%.

Expected

• No extension write command, and no auto-resolve, ever overwrites entries it did not change in the shared lockfile, including from a fresh or stale cache.
• Auto-resolved extensions are recorded in the shared lockfile and published.
• Files already on disk are adopted instead of dead-ending.
• The legacy in-repo lockfile is retired safely, and #2483's transitional read is removed.