Skip to main content
← Back to list
01Issue
BugOpenExtensionsPublic
AssigneesNone

Relationships

#1771 pullChanged skips same-size updated files — size-only change detection ignores the index sha256

Opened by psftw · 8/21/2026

pullChanged decides whether a file changed by size alone: a local file whose byte length matches the index entry is never re-downloaded, so an in-place update that preserves length is invisible to every reader, forever. Expected: pull uses the change detection the push side already maintains — the index entry carries a sha256 precisely so same-size changes are detectable.

In datastores/_lib/s3_cache_sync.ts (2026.08.12.1), the pullChanged walk:

const stat = await Deno.stat(localPath);
if (stat.size === entry.size) {
  // File exists locally with matching size — no download needed.
  ...
  continue;
}

while the push side (fileNeedsPush) fingerprints size → mtime → sha256 and writes a fresh sha256 into the shared index on every upload. The asymmetry means a writer correctly publishes a same-size update (new bytes, new sha256 in the index) and every reader already holding an old same-size copy skips the download on every future sync. The pull walk even reconciles the index entry's localMtime to the reader's own file, suppressing the drift signal.

Repro (two clones, A and B, sharing one bucket):

A: write a synced file with content AAAA; datastore sync   # push
B: datastore sync                                          # B has AAAA
A: change the content to BBBB (same length); datastore sync
B: datastore sync

Expected: B reads BBBB. Actual: B reads AAAA, and no number of syncs repairs it — B stays stale until the file's size happens to change.

Upstream repository: https://github.com/systeminit/swamp-extensions

Environment

  • Extension: @swamp/s3-datastore@2026.08.12.1
  • swamp: 20260820.072114.0-sha.05d20b6a
  • OS: linux (x86_64)
  • Deno: 2.8.3
  • Shell: /bin/bash
02Bog Flow
◉OPEN○TRIAGED○IN PROGRESS○SHIPPED

Open

8/21/2026, 5:58:31 PM

No activity in this phase yet.

03Sludge Pulse

Sign in to post a ripple.