Skip to main content
← Back to list
01Issue
BugShippedSwamp CLIPublic
Assigneeshammz

Relationships

#2483 managedConfig: fix startup config-base resolution and extension list (prerequisites for swamp-club#2429)

Opened by hammz · 9/24/2026· Shipped 9/28/2026

Description

Split from swamp-club#2429. That issue moves pulled extension sources into the datastore config tier under managedConfig: true. The four bugs below must be fixed first. None of them moves any files. All four were found while triaging #2429 (repro scratch repo: filesystem datastore outside the repo, then swamp datastore config migrate, swamp 20260923.213824.0-sha.edd0e85e).

1. Startup records a guessed config base for extension-backed datastores

src/cli/mod.ts:1545 calls ensureManagedConfigBase before configureExtensionLoaders (1550) and before the auto-resolver is set (~1653). At that point the datastore type registry has no loader (datastore_type_registry.ts:99) and there is no auto-resolver. So for an S3/GCS datastore, resolveDatastoreConfig throws "Unknown datastore type". repo_context.ts:291-300 swallows the error at debug level. resolveManagedConfigPaths (repo_context.ts:322-334) then registers the guessed <repo>/.swamp/config as the managed config base.

Consequences today:

  • Wrong lockfile at startup. The model, vault, report and webhook loaders, the ExtensionRepository and the auto-resolver capture the in-repo lockfile <repo>/.swamp/config/upstream_extensions.json (mod.ts:404-411, 1546-1557, 1651-1660). The real lockfile is <cache>/<ns>/config/upstream_extensions.json.
  • Warning reads the same file. The startup "N pulled extension(s) have missing source files" warning (mod.ts:1128-1142) reads the same in-repo lockfile.
  • Registry changes mid-run. requireInitializedRepo* and the extension commands later overwrite the registry with the real tier (repo_context.ts:575/823/991). The registered base therefore changes during a single process.
  • Silent fallback on failure. When datastore resolution fails inside an extension command (offline, no trusted collective, or the #445 case of pulling the datastore extension itself), pull and install silently fall back to the in-repo base (extension_pull.ts:280-284, create_extension_install_deps.ts:58-59).

Filesystem datastores are unaffected: they resolve without loading an extension (resolve_datastore.ts:402-424).

2. extension list ignores managedConfig

createExtensionListDeps (src/libswamp/extensions/list.ts:59-71) always reads <modelsDir>/upstream_extensions.json. It is used by the CLI (extension_list.ts:170) and by serve (admin_handlers.ts:454). In a managed repo, swamp extension list --json returns {"extensions": []} both locally and through --server, even though the extension is installed and model type describe resolves it.

3. Stale catalog rows crash model type describe

Repro:

  1. Pull an extension (for example @swamp/aws/cur).
  2. Run swamp datastore config migrate.
  3. Run swamp extension install. It reports orphans_pruned and then installed: 1, migrates the legacy files and sweeps the old copy.
  4. Run swamp model type describe @swamp/aws/cur/report-definition.

Step 4 exits 1 with "No such file or directory (os error 2): readfile '/.swamp/pulled-extensions/@swamp/aws/cur/models/report_definition.ts'". The stack runs through ExtensionLoader.recoverMissingBundle. The cause: extension install uses the free-function install path, which has no catalog (create_extension_install_deps.ts:79-91). sweepLegacyPaths deletes the legacy sources, but their catalog rows stay Indexed.

After swamp extension rm, describe still crashes with ENOENT, this time on the .swamp/config/pulled-extensions/... path, because a catalog row survived the rm. rm tombstones only rows matched by extension_name, and path-based identity (derive_extension_identity.ts:109) does not recognise .swamp/config/pulled-extensions/. Lazy registration should also skip, with a warning, an Indexed row whose source no longer exists, instead of failing the whole command.

4. extension rm changes state before validating, and deletes by lexical containment only

RemoveExtensionService.execute (remove_extension_service.ts) tombstones the catalog (~120) and removes the lockfile entry (~138) before assertContainedPath runs on each tracked file (~147). A rejected entry therefore leaves the catalog and lockfile emptied while the files stay on disk, and a retry says "Extension X is not installed". Every tracked file should be resolved and validated before anything changes.

Containment is also lexical and repo-wide:

  • Cross-extension deletes. An entry for extension A can list extension B's files, or .git/... and .swamp.yaml, and rm deletes them.
  • Symlinks. assertContainedPath (safe_path.ts:129-160) does not follow symlinks.
  • Unbounded pruning. pruneEmptyDirs (~220-246) stops on a path-length comparison rather than containment.

The lockfile is repo-controlled input and, under managedConfig, is synced from a shared remote. Each entry should be contained to its extension's own roots (pulled subtree, bundle dirs, skill dirs, legacy prefixes), with symlink-aware checks at the delete sinks.

Expected

  • Startup: the managed config base is resolved once, correctly, before any loader captures a lockfile path, or the loaders read it when they load. The registry records whether the base was resolved or guessed. Write commands warn, or fail, when they would fall back to a guessed base under managedConfig with an extension-backed datastore.
  • extension list: it reads the managed lockfile in both CLI and serve.
  • Catalog: no stale Indexed rows survive an install migration or an rm, and a stale row never crashes a command.
  • rm: it validates every entry against per-extension roots before changing anything.
  • Tests: add a regression test with an extension-backed datastore type, since a filesystem datastore cannot catch bug 1.

These changes do not move pulled sources; swamp-club#2429 does that.

02Bog Flow
✓OPEN✓TRIAGED✓IN PROGRESS✓SHIPPED+ 1 MOREASSIGNED+ 20 MOREREVIEW+ 36 MOREPR_MERGED+ 2 MORESESSION_SUMMARIZED

Shipped

9/28/2026, 3:00:22 PM

Click a lifecycle step above to view its details.

03Sludge Pulse
hammz assigned hammz9/24/2026, 2:22:54 PM
Editable. Press Enter to edit.

hammz commented 9/24/2026, 3:59:41 PM

Split on 2026-09-24 during triage. This issue now covers only bug 1 (startup records a guessed config base for extension-backed datastores) and bug 2 (extension list ignores managedConfig). Bug 3 (stale catalog rows crash describe) moved to swamp-club#2490. Bug 4 (rm validates after mutating, lexical-only containment) moved to swamp-club#2489 together with the same hardening for install's orphan prune and legacy sweep. #2489 is a security issue and should ship first, because a committed lockfile entry listing src plus swamp extension install deletes src/ today. All three block swamp-club#2429. Design decisions for this issue: loaders read the union of the shared lockfile (the resolved managed config base, synced) and the instance-local in-repo lockfile, with the shared entry winning; the auto-resolver keeps installing into the local lockfile; explicit extension commands write the shared lockfile behind a resolution guard. The adversarial findings for bugs 3 and 4 are recorded on this issue's lifecycle and summarised in the new issues.

hammz commented 9/24/2026, 5:05:20 PM

Second split on 2026-09-24. Writing the managed extension lockfile moved to swamp-club#2495: hydrate before write (the lost-update bug, where a fresh or stale cache overwrites the team's lockfile), auto-resolve recording and publishing to the shared lockfile, adopting extension files already on disk, and retiring the in-repo lockfile. The design review showed that work needs its own global-lock, sync-session and serve-gate design. This issue keeps startup resolution of the managed config base (datastore extensions discovered on disk, installed-only resolution, provenance in the registry), the guard that refuses extension writes to a guessed base, and extension list reading the managed lockfile. Interim until #2495 lands: auto-resolve keeps recording installs in the in-repo lockfile as today, and loaders, reconcile and list read that file read-only alongside the shared lockfile.

hammz commented 9/24/2026, 9:19:46 PM

Split for review into four staged PRs, merged in this order: (1) swamp-club#2507 records whether the managed config base was resolved and adds installed-only resolution; (2) swamp-club#2508 makes extension list read the managed lockfile (this issue's bug 2); (3) this issue resolves the managed config base at startup (on-disk datastore discovery, lazy loader resolution, the transitional in-repo auto-resolve lockfile with pins read from the team lockfile); (4) swamp-club#2509 adds the write guard and the JSON skip reporting. Each PR is verified and attested on its own. The rm/update gap for the in-repo auto-resolve lockfile stays with swamp-club#2495.

hammz commented 9/25/2026, 2:42:50 PM

Part 3 split again for review (2026-09-25), merged in this order: swamp-club#2529 (3a, buffer startup warnings) -> swamp-club#2530 (3b, pulled sources always in the repo's pulled root) -> swamp-club#2531 (3c, find datastore extensions on disk) -> swamp-club#2532 (3d, transitional in-repo auto-resolve lockfile, read side) -> this issue (3e, resolve the managed config base at startup) -> swamp-club#2509 (write guard). Each is verified and attested on its own.

Sign in to post a ripple.