Relationships
#2907 S3/GCS datastore: a push that fails after uploading drops its recorded deletes, so the retry brings deleted data back
Opened by stack72 · 10/1/2026
Problem
In @swamp/s3-datastore and @swamp/gcs-datastore (still true in 2026.10.01.1), a push that fails after its uploads have started sets the bulk-invalidated flag. The next push is then a full walk, and a full-walk push deletes nothing (the bulk-disables-deletes behaviour, s3_cache_sync.ts around S3SYNC:2822 and 3095-3165). Any per-path deletes recorded for that cycle are lost.
Result: if one cycle both writes item kept and deletes a previously published item gone, and the push fails after uploading, the retry publishes kept but never deletes gone. The deleted item stays on the remote, and every peer keeps reading it (it comes back for anyone who pulls).
Reproduction
Pinned in swamp integration/datastore_remote_failure_test.ts (swamp-club#2859) against the in-memory remote, which models the extensions: write one item and delete a published one in the same cycle, failNext push with afterUploads, then retry the flush through acquireModelLocks or flushDatastoreSync. pendingPush shows deletes recorded before the failure and an empty delete list with bulk true after it; after the retry, peer B still reads the deleted item.
Expected
A failed push keeps its recorded deletes (or the full-walk retry can still delete), so the retry applies them.
Related
Tracking swamp-club#2865. Related to swamp-club#2888 (failed push kept dirty through a pull), which did not change this path.
Open
No activity in this phase yet.
system commented 10/1/2026, 5:26:35 PM
Classified automatically when this issue was filed.
- Source: Extensions
If you feel this classification is incorrect, add a ripple to tell us so.
Sign in to post a ripple.