#1458 s3-datastore: bidirectional sync pull writes namespace-stripped copies to cache root, wedging push with no supported recovery

Summary

With both prefix and namespace configured, swamp datastore sync (bidirectional) pulls objects from <prefix>/<namespace>/… and writes them into the local cache at the root (solo) layout instead of under <cache>/<namespace>/. The push half of the same command then hard-errors on the very files the pull just created.

swamp datastore namespace migrate --confirm — the recovery the error message recommends — silently no-ops when the destination files already exist, so there is no supported way out.

Environment

• swamp 20260728.221558.0-sha.f1005334
• @swamp/s3-datastore 2026.07.25.1 (latest at time of filing)
• macOS, Darwin 25.5.0, arm64
• Datastore config: {"bucket":"…","region":"us-west-2","prefix":"swamp"}, namespace homelab

Reproduction

1. Repo on the S3 datastore with both a prefix and a namespace. All data correctly under <prefix>/<namespace>/ in S3; local cache clean (contains only <cache>/<namespace>/).
2. swamp doctor datastores → overallStatus: pass (all three checks).
3. swamp datastore sync --push -y → {"mode":"push","filesPushed":0} — converged.
4. swamp datastore sync ← bidirectional
5. Observe: [s3-sync] Skipped N solo-layout file(s) outside the bound namespace "homelab". These are stale leftovers — investigate and remove them. together with filesPulled: N. The cache now contains root-level data/, outputs/, auto-definitions/, definitions-evaluated/, workflow-runs/, workflows-evaluated/ alongside homelab/.
6. swamp datastore sync again → hard error: Cannot push: un-migrated data found at root level (auto-definitions, definitions-evaluated, workflows-evaluated, data, outputs, workflow-runs). Run 'swamp datastore namespace migrate' to preview, then --confirm to move data under the "homelab" namespace before pushing.
7. swamp datastore namespace migrate --confirm → prints the preview file listing and exits without migrating, because the destinations already exist under homelab/. Root dirs remain; push still errors. Repeating step 7 changes nothing.

Expected

A namespace-scoped pull should write to <cache>/<namespace>/…, matching the layout push expects. With local and remote already in agreement, swamp datastore sync should be a no-op.

Actual

Pull writes namespace-stripped duplicates to the cache root; push rejects them; the documented recovery cannot merge into a non-empty destination. The repo is wedged for all pushes until the root-level directories are manually removed from ~/.swamp/repos/<repoId>/.

Notes

• The root-level files are byte-identical duplicates of their counterparts under homelab/ (verified with cmp).
• Reads are unaffected — swamp model list and swamp data list keep working and resolve the correct namespace throughout. Data integrity was never at risk in my case.
• swamp datastore sync --push alone is unaffected, and is a usable workaround.
• swamp doctor datastores does correctly detect the bad state (overallStatus: fail, namespace_migration: passed: false).
• However swamp doctor datastores --repair returns {"status":"not_needed"} — it targets foreign-namespace contamination (see #1353), not root-level un-migrated data, so it does not recover this.
• Only a manual rm/mv of the root-level dirs in the cache restores a pushable state.

How a clean setup reaches this state

swamp datastore setup extension @swamp/s3-datastore --namespace <ns> … that fails partway through — in my case the credential preflight timed out (PREFLIGHT_TIMEOUT_MS) — still moves local .swamp/ data into the cache at solo layout before erroring. A successful re-run then binds the namespace and correctly refuses to absorb those files. namespace migrate --confirm folds them in (this works while <ns>/ is still empty), and the subsequent push succeeds.

The first bidirectional swamp datastore sync after that re-creates the root layout — and from then on recovery via supported commands is impossible.

Suggested fixes

1. Namespace-scoped pull should write under <cache>/<namespace>/, symmetric with push.
2. namespace migrate --confirm should either merge into a non-empty destination (skipping identical files) or fail loudly explaining why it cannot, rather than printing a preview and exiting 0.
3. Consider extending doctor datastores --repair to cover root-level un-migrated data, since it already detects it.

Upstream repository: https://github.com/systeminit/swamp-extensions

Environment

• Extension: @swamp/s3-datastore@2026.07.25.1
• swamp: 20260728.221558.0-sha.f1005334
• OS: darwin (aarch64)
• Deno: 2.8.3
• Shell: /bin/zsh