Skip to main content
← Back to list
01Issue
BugShippedExtensions
Assigneesstack72

Relationships

#913 datastore migrate-index ignores the repo's configured namespace (migrates bucket-root index instead)

Opened by mgreten · 7/2/2026· Shipped 7/2/2026

swamp datastore migrate-index ignores the repo's configured namespace — migrates the bucket-root index instead

First off — thank you for the shard-first work, and for crediting the measurements. The migration path itself is lovely: explicit migrate-index command (no accidental migration), clean _meta.json v2 + commitSeq, and the monolith→shard partitioning ran in ~8s. I hit one thing while migrating a namespaced repo and wanted to share a clean repro, in case it's useful — very possibly I'm holding it wrong.

Environment

  • swamp 20260702.005505.0-sha.2424f433
  • @swamp/s3-datastore@2026.07.01.1, MinIO backend
  • Repo .swamp.yaml has datastore.namespace: agentic-tooling (shared bucket, multiple repos each under their own namespace — the multi-writer case shard-first targets)

What happens

Running swamp datastore migrate-index from a repo whose datastore is namespaced appears to migrate the bucket-root index (_index/_meta.json), not the repo's namespaced index (<namespace>/_index/_meta.json). Concretely, after migrating:

root  _index/_meta.json:            version 2, 13 partitions   ← migrated
agentic-tooling/_index/_meta.json:  version 1, 289 partitions  ← NOT migrated (still monolithic)

Every migrate-index run reports 299321 entries → 13 shard(s) — that's the bucket-root index's entry count. The live namespaced index (agentic-tooling/.datastore-index.json, ~229 MB, the one all my model writes actually update) stays v1.

Downstream: because the namespaced _meta never flips to v2, commitPush keeps taking the monolith path and logs:

[s3-sync] Index is using monolithic format. Run 'swamp datastore sync --migrate-index' to enable shard-first commits...

and the index-merge under the lock stays ~60s — so on a namespaced repo the shard-first benefit isn't reachable via the CLI. SWAMP_NAMESPACE=<ns> in the env didn't change the behavior either.

Where I think it is (hypothesis, defer to you)

migrateMonolithToShards in s3_cache_sync.ts is itself namespace-aware — it goes through metaKey()/shardKey()/writePartitionMeta, which all branch on this.namespace. So the service is fine. It looks like the CLI migrate-index command may be constructing/using the sync service without binding the repo's namespace onto it first (the way sync and model-method writes do), so this.namespace is empty and everything resolves to bucket-root. If that's right, binding the configured namespace before calling migrate would fix it — or a --namespace flag on the command as an escape hatch.

Blast radius / safety

No data harmed here — the migration only ever touched the (stale, unused) bucket-root index; the live namespaced data is untouched and writes still work normally on the monolith path. So this is "the benefit isn't reachable for namespaced repos," not "it broke something." I backed up first out of caution.

Happy to share the full _meta.json bodies, the migrate --verbose output, or test a build against the namespaced repo whenever it helps. Thanks again — this is the piece that closes out the whole #812/#829 arc for us once namespaced repos can reach it.

02Bog Flow
✓OPEN✓TRIAGED✓IN PROGRESS✓SHIPPED+ 2 MORETRIAGE+ 6 MOREREVIEW+ 4 MOREPR_MERGED+ 1 MORECONTRIBUTOR_NOTIFIED

Shipped

7/2/2026, 4:24:31 PM

Click a lifecycle step above to view its details.

03Sludge Pulse
stack72 assigned stack727/2/2026, 3:21:38 PM
Editable. Press Enter to edit.

stack72 commented 7/2/2026, 4:29:12 PM

Thanks @mgreten for reporting this! The fix has been merged and a release is on its way. We appreciate your contribution to swamp.

Sign in to post a ripple.