Relationships
#3707 Catalog rebuild restores pre-rename versions of a name renamed away and written again
Opened by zack · 10/21/2026
Summary
A catalog rebuilt from disk (full backfill) and the write-through catalog disagree about the history of a data name that was renamed away and then written again. The rebuild brings back the versions from before the rename; write-through had removed them.
Steps to reproduce
With FileSystemUnifiedDataRepository and a populated write-through CatalogStore:
savedata namea(v1).renameatob— write-through removesa's rows, writesb@1, records the forwarda -> b, anda@2on disk is the rename marker.saveaagain (v3) — write-through upsertsa@3and ends the forward.- Rebuild a fresh catalog from the same files (
DataQueryService.queryon an unpopulated catalog).
Actual
- write-through rows:
a@3 (latest),b@1 (latest) - rebuilt rows:
a@1,a@3 (latest),b@1 (latest)
The backfill indexes every version of a from listVersions, skipping only the rename marker version, so the pre-rename a@1 returns as a non-latest history row.
Expected
A rebuilt catalog matches the write-through catalog. Either the backfill drops versions below the most recent rename marker of a name, or write-through keeps them; pick one and make both paths agree.
Impact
Latest rows match, so data.latest(), default queries and the swamp doctor datastores completeness check are unaffected. History queries (isLatest == false, version == 1) return a pre-rename version only on rebuilt catalogs.
Found while writing the rebuild-equals-write-through property test for swamp-club#3698; that test's generator excludes re-saving a renamed-away name because of this.
Closed
No activity in this phase yet.
zack commented 10/21/2026, 2:42:16 PM
Closing. Triage found that this affects only v2 datastores.
Reproduced on a v2 filesystem datastore with the CLI. Write data name a, run swamp data rename to move it to b, write a again, then delete .swamp/data/_catalog.db and query. The rebuilt catalog has an extra non-latest a@1 row that the write-through catalog never had. Latest rows match on both sides.
The extra row comes from the v2 full backfill (DataQueryService.backfillAsync and backfillSync), which indexes every version on disk except the marker versions. v3 datastores never run that backfill: repository_factory.ts creates the query service with backfill turned off when the v3 read model supplies the catalog. A v3 catalog is rebuilt by replaying commits or loading the compacted catalog state. Those paths drop the old name's rows on a rename and add back only the version written afterwards, which is what write-through does. The v3 conclusion comes from reading the code; it was not reproduced on a v3 datastore.
Sign in to post a ripple.