#2429 managedConfig: serve extension handlers disagree on the pulled-extensions root, and lockfile files[] are repo-relative

Reproduced on main @ 1da85ec8 (20260923.175528.0-sha.1da85ec8, unmodified) with @swamp/s3-datastore@2026.09.23.1 on local MinIO, managedConfig: true, hydrationStrategy: lazy. Every instance started with an empty HOME and a repo containing only .swamp.yaml, like a stateless pod. The results below extend this issue in three ways.

1. With a remote datastore, pulled files never reach the datastore

The split isn't only between handlers. extension pull extracts under <repo>/.swamp/config/pulled-extensions/, but managed-config sync pushes getManagedConfigBase() ($SWAMP_HOME/repos/<repoId>/config/). The lockfile entry is uploaded and the files aren't:

$ find inst1/repo/.swamp/config inst1/home/.swamp/repos -path '*cve*' -name manifest.yaml
repo/.swamp/config/pulled-extensions/@swamp/cve/researcher/manifest.yaml

$ jq '."@swamp/cve/researcher".files' <bucket>/config/upstream_extensions.json
[".swamp/config/pulled-extensions/@swamp/cve/researcher/manifest.yaml", …]

bucket config/ after the pull:
  config/managed-config-migrated.json
  config/pulled-extensions/@swamp/s3-datastore/…
  config/upstream_extensions.json
  (nothing under config/pulled-extensions/@swamp/cve/researcher/)

Consequences:

• A second, fresh serve instance on the same datastore returns Model type not found for the pulled type.
• The same instance, restarted on its own volume, also loses it. It ends up with two lockfiles that disagree:

  repo/.swamp/config/upstream_extensions.json            -> @swamp/s3-datastore
  home/.swamp/repos/<id>/config/upstream_extensions.json -> @swamp/cve/researcher, @swamp/s3-datastore
  after restart: @swamp/cve/researcher MISSING

This affects the CLI too: swamp extension pull from an operator repo (no --server) also extracts to the repo path and uploads nothing.

With the filesystem datastore the cache is <repo>/.swamp, so both paths are the same and none of this shows.

2. On main, serve's pull also writes to the repo root

The description says serve's extension.pull resolves through resolveManagedPathsFromContext to the cache root. In our run, swamp extension pull @swamp/cve/researcher --server <url> extracted to <repo>/.swamp/config/pulled-extensions/ (output above), and nothing appeared under $SWAMP_HOME/repos/<id>/config/pulled-extensions/. Either the cache root isn't being picked up in the handler, or something else resolves the extract target. Worth checking when fixing this.

3. The files[] containment failure reproduces

Once pulled files live under the managed base, lockfile entries point outside repoDir. We rewrote the entries of a pulled package to the paths a fix would record, then ran swamp extension install:

Path traversal detected: "../operator-home/.swamp/repos/x/config/pulled-extensions/@swamp/git/files/LICENSE.txt" resolves to …

That's libswamp/extensions/install.ts, needsInstallOrMigration() → assertContainedPath(file, repoDir) (L332, L381). With the unmodified repo-relative entries, install succeeds. So any fix that moves the files has to change containment at the same time: check against the managed base, or store entries relative to it.

Suggested scope for the fix

Resolve the pulled-extensions root and the managed lockfile path from getManagedConfigBase() for every caller:

• pull (CLI and serve), install, update and rm
• the model loader, the extension discoverer and reload

The same approach already works for vaults: resolveEffectiveVaultsDir() in paths.ts, #2403. Today resolvePulledExtensionsRoot() (L176–182) and managedConfigLockfilePath() (L189–191) both hard-code swampPath(repoDir, "config", …).

Workaround in the meantime: symlink or mount $SWAMP_HOME/repos/<repoId> → <repo>/.swamp so both paths are the same directory. With that in place, pulled files upload and hydrate.

Full repro scripts and transcripts are available if useful.

Follow-up from swamp-club#2415, which removes the bare markDirty() calls from the serve extension handlers.

Under managedConfig, those handlers now mark only the config-tier lockfile (upstream_extensions.json) before pushing. Serve writes extension sources to the repo-local pulled-extensions root, so the lockfile is the only file they change in the datastore cache. A directory mark on the tier root would walk a tree the handler never wrote. That could delete, or overwrite with stale local copies, extensions another serve instance pushed and this one has not polled yet.

Once this issue makes install, pull, update and rm write the datastore-tier root, they also need per-path directory marks:

• one mark on pulledExtensionsRoot/ for each extension the operation installed, updated or removed;
• for pull, install and update, dependencies too, walking dependencyResults recursively (install.ts and update.ts currently discard them);
• a check of every name with validateExtensionName before it is joined into a path, since names come from the lockfile.

Auto-resolved installs (vault create/migrate, model create) have the same gap: their extension files and lockfile change are never marked. See the Serve handler obligation section in design/enablers/datastores.md.

Triage update. This issue now covers moving pulled extension sources into the datastore config tier. Its prerequisites were split out to swamp-club#2483, which must land first. Retriage this issue after that.

Triage corrected the premise. In practice, CLI and serve pull, install, update and rm all agree on the repo-local .swamp/config/pulled-extensions. The explicit root is dropped at the pull call sites (extension_pull.ts:321, the serve pullDeps), and resolvePulledExtensionsRoot ignores the registered config base. Only the lockfile reaches the tier, so sources never sync. Nothing produced ../ entries in files[]. This isn't a regression: the plumbing from #2048 was dropped at its own call site.

A plan-v1 adversarial review raised points the retriage must cover:

• Bootstrap exception. Extensions that provide the marker's datastore type, and their dependencies, cannot live only in the tier they define. Keep them repo-local, as datastore-bundles is kept (datastore_config.ts:72-74).
• Lockfile meaning. Reusing the .swamp/config/pulled-extensions/ prefix as a logical path changes what existing S3/GCS entries mean. That affects mixed-version fleets and the cleanup of old repo copies, since sweepLegacyPaths skips current-layout entries.
• Relocation guard. Relocate only when the tier was really resolved, not guessed (this depends on #2483).
• Delete sinks. Contain each entry to its extension's roots with symlink-aware checks. Stop rm's pruneEmptyDirs at the extension root.
• Name attribution. #checkAuthorization allows the call when the caller has no extensionName. Attribute the name from the lockfile or catalog, and fix the extractExtensionNameFromPath realpath early return and the model_invocation_service.ts:501 fallback.
• Layering. Put the config-migrate lockfile rewrite in the CLI or libswamp, not in managed_config_migration.ts (domain).
• Tests. Include an integration case with an extension-backed datastore type. File swamp-uat issues for managedConfig S3/GCS extension lifecycle coverage (none exists today).