Skip to main content
← Back to list
01Issue
BugShippedExtensions
Assigneesstack72

Relationships

#797 pushChanged does not implement absence-on-disk deletion (markDirty contract rule #2)

Opened by stack72 · 6/24/2026· Shipped 6/24/2026

Summary

pushChanged in S3CacheSyncService does not delete remote S3 objects when locally-deleted files are detected via the markDirty contract's rule #2 (absence-on-disk = delete).

When core calls markDirty(relPath) before removing a file, the sync service records the path as dirty. On the next pushChanged, the scoped walk stats each dirty path — but when the path is absent (because it was deleted), the catch block silently ignores it instead of issuing an S3 DELETE:

```ts } catch { // Path may have been deleted between markDirty and push } ```

For the bulk walk path (triggered when >200 dirty paths overflow DIRTY_PATHS_CAP), the full walk only iterates local files and never compares the index against local state to find entries that were deleted.

Context

swamp-club#788 fixed the core side: swamp data gc now emits per-path notifyDirty(versionDir) calls for each version being removed (PR swamp-club/swamp#1667). The extension needs to act on those signals.

Required Changes

  1. Scoped walk: when a dirty path is absent locally, find index entries matching that prefix and collect them for S3 deletion
  2. Bulk walk: after walking local files, iterate the index to find entries with no local counterpart — collect those for deletion (guarded by !lazyPullActive to avoid deleting un-hydrated content)
  3. Issue S3 DELETEs in batches (matching the existing push concurrency), remove deleted entries from the index, and include them in the index writeback condition
  4. Include deleted paths in the partition key tracking for partitioned index updates

Backwards Compatibility

The markDirty contract already defines rule #2 as SHOULD. This change makes the extension implement the rule it previously acknowledged but did not act on. No interface changes needed.

Upstream repository: https://github.com/systeminit/swamp-extensions

Environment

  • Extension: @swamp/s3-datastore@2026.06.03.2
  • swamp: 20260623.195403.0-sha.89da3589
  • OS: darwin (aarch64)
  • Deno: 2.8.3
  • Shell: /bin/zsh
02Bog Flow
OPENTRIAGEDIN PROGRESSSHIPPED+ 1 MOREASSIGNED+ 2 MOREREVIEW+ 3 MOREPR_MERGED+ 1 MORENOTIFICATION_SKIPPED

Shipped

6/24/2026, 3:17:21 PM

Click a lifecycle step above to view its details.

03Sludge Pulse
stack72 assigned stack726/24/2026, 2:19:09 PM

Sign in to post a ripple.