Skip to main content
← Back to list
01Issue
BugIn ProgressSwamp CLI
Assigneesstack72

Relationships

#1304 namespace unset --migrate silently deletes artifact history via catalog-rebuild GC (~484k files lost)

Opened by magistr · 7/20/2026

Summary

swamp datastore namespace unset --migrate --confirm invalidates the data catalog. On the next access the catalog is rebuilt, and the rebuild applies each spec's garbageCollection: N retroactively to history that predates it — silently deleting every version beyond the retention count.

On a production repo this destroyed ~484,000 files / ~6 GB of artifact history: summaryMain went from 1,300 retained versions to 10, and per-apartment artifacts were cut to their declared 5. There was no warning, no prompt, and no mention in the command's output.

This is a data-loss bug, not a display issue. The versions are gone from disk.

What made it surprising

The reverse migration reports success in terms of files preserved:

Reverse migration complete: 11 directory(s) moved
  Total: 793007 file(s), 15464.8 MB

Immediately afterwards the same tree measured 301,998 files / 3.8 GB. So the command truthfully reported moving 793k files and then, via the catalog invalidation it also performs, caused most of them to be deleted. Nothing in the output connects those two events.

The retention limits themselves had never been enforced before — swamp data gc is a manual command and had not been run on this repo, so 1,300 versions had accumulated under a garbageCollection: 10 spec. Enforcement therefore arrived as a one-time bulk deletion triggered by an unrelated layout operation.

Reproduction sketch

  1. Repo with a filesystem datastore and a spec declaring garbageCollection: 10.
  2. Accumulate substantially more than 10 versions of that resource (do not run swamp data gc). Confirm with swamp data versions <model> <name>.
  3. swamp datastore namespace set <ns> then swamp datastore namespace migrate --confirm.
  4. swamp datastore namespace unset --migrate --confirm.
  5. swamp data versions <model> <name> now reports 10. The version directories for the rest are removed from disk.

Expected

Any operation that can delete artifact history should say so before doing it. Concretely, at least one of:

  • Do not run GC as a side effect of a catalog rebuild. Leave enforcement to the explicit swamp data gc.
  • If the rebuild must reconcile retention, report it — "N versions exceed retention and will be deleted" — and require confirmation, the way swamp data gc itself does.
  • Have namespace migrate / unset --migrate state up front that it invalidates the catalog and what the consequence of that is.

The asymmetry is what stings: namespace migrate has a careful preview mode and a --reverse escape hatch, which reads as "this operation is safe and reversible". The irreversible part is the catalog side effect, which has neither.

  • swamp-club#1265 — namespace migrate/set broken for extension datastores
  • swamp-club#1280 — sync guard skips bound namespace on push
  • swamp-club#1172 — no retention/prune for workflow-runs and outputs

Context: this was hit while trying to move a repo onto a MongoDB datastore. The namespace migrate turned out to be the wrong operation for an extension datastore anyway (per #1265, "the local tier is never namespaced"), and the reverse was an attempt to undo it. Worth noting that namespace set actively recommends running migrate, so this path is easy to walk into.

Environment

  • swamp: 20260720.142822.0-sha.0caa5556
  • datastore: filesystem
  • OS: linux (x86_64), Debian
  • Tree: ~15.5 GB, ~793k files before; ~3.8 GB, ~302k files after
02Bog Flow
OPENTRIAGEDIN PROGRESSSHIPPED+ 1 MOREASSIGNED+ 5 MOREREVIEW+ 3 MOREPR_LINKED

In Progress

7/21/2026, 4:08:37 PM

Click a lifecycle step above to view its details.

03Sludge Pulse
stack72 assigned stack727/20/2026, 11:31:54 PM

Sign in to post a ripple.