← Back to list
01Issue
BugOpenSwamp CLIPublic
AssigneesNone
Relationships
#2892 Extension datastore: query still returns an item after its delete is pulled
Opened by stack72 · 10/1/2026
Summary
On an extension datastore (S3, GCS), after a peer's delete reaches a repo through a pull, DataQueryService.query in that repo still returns the deleted item. getLatestRecord correctly returns null.
Found while working on swamp-club#2856 / #2858.
Reproduction (in-process, in-memory remote)
- Repos A and B on one
createInMemoryRemote({ semantics: { pullDeletes: true } })(the default fake reproduces the extensions' current pull, which does not delete local files at all, see note below). - A saves item
x, pushes. B pulls, queries:[x v2]. - A deletes
x, pushes. B pulls (synced=true), invalidates the catalog. - B
getLatestRecord("m1", "x")returns null, butquery('modelName == "m1" && name == "x"')still returns[2].
Why
- Full backfill is additive by design (swamp-club#1581): it upserts rows found on disk and never removes rows, so rows for remote-only metadata under lazy hydration survive.
- Custom datastores construct
DataQueryServicewithfilterStaleRows: false(repo_context.ts), soquerydoes not hide rows whose files are gone; filesystem datastores set it to true. - So the deleted item's row survives the backfill and is returned.
Notes
- The obvious fix (replace this namespace's rows during a full backfill) fails
DataQueryService: backfill preserves catalog rows the walk cannot seeand reintroduces #1581. A fix needs to tell remote-only rows (lazy hydration) apart from rows whose data was deleted, for example by pruning rows whose metadata is gone locally only when the datastore does not use lazy hydration, or by a tombstone-aware pull. - Separately, the S3/GCS extensions' pull never deletes local files (
pullDeletes: falsein the in-memory remote mirrors this), so in practice a peer's delete does not reach other repos' caches at all. That part lives in swamp-extensions.
02Bog Flow
Open
No activity in this phase yet.
03Sludge Pulse
Sign in to post a ripple.