Skip to main content
← Back to list
01Issue
FeatureClosedSwamp CLIPublic
AssigneesNone

Relationships

#2362 doctor datastores: report local mirror files the remote no longer has

Opened by stack72 · 9/22/2026

Follow-up to swamp-club#2353 (deferred from its core plan until the S3/GCS extension fix settles the design).

Problem

On S3/GCS-backed repos a machine's local mirror can hold files that were deleted remotely. Pulls never remove them, and push walks re-upload them, silently undoing data delete/gc/prune. Nothing in the CLI shows this condition: data list, data versions and data gc --dry-run all read the local index and report the cleanup as complete. The reporter of #2353 only found it by diffing the mirror against the bucket listing by hand.

Proposal

  1. Add an optional method to the DatastoreSyncService port, duck-typed like repairNamespaceContamination, that reports local mirror files absent from a complete remote listing and without a pending local change. A pending local change means a path in the dirty set or, on a bulk walk, a file modified after the last completed sync. The method returns a count, sample paths, and a skipped reason when the extension cannot tell stale files from unpushed ones. It must never report on a partial or failed listing.
  2. Add a mirror-consistency check to swamp doctor datastores, in log and json modes and in the serve admin handler, that shows the result or not-supported.
  3. Repair (removing stale local files) is optional and needs care: doctor datastores takes no lock today, so a non-dry-run repair must hold the global datastore lock, keep files within the cache root, and refuse while unpushed changes exist.
  4. Add a two-mirror behavioural conformance suite in packages/testing covering: a remote deletion is pruned from a peer's mirror on pull; a bulk-walk push does not resurrect it; a directory relPath removes every file under it remotely; a genuinely new local file is still pushed; a failed listing never prunes; and an already-corrupted mirror (a stale file in neither index) is neither uploaded nor kept.

Dependency

Needs the S3/GCS extension changes from #2353 (pull-side pruning and the pending-local-change rule) so the port contract matches a working implementation before it is published.

02Bog Flow
✓OPEN○TRIAGED○IN PROGRESS◉CLOSED

Closed

9/22/2026, 9:40:00 PM

No activity in this phase yet.

03Sludge Pulse

Sign in to post a ripple.