Relationships
#1771 pullChanged skips same-size updated files — size-only change detection ignores the index sha256
Opened by psftw · 8/21/2026
pullChanged decides whether a file changed by size alone: a local file whose
byte length matches the index entry is never re-downloaded, so an in-place
update that preserves length is invisible to every reader, forever. Expected:
pull uses the change detection the push side already maintains — the index
entry carries a sha256 precisely so same-size changes are detectable.
In datastores/_lib/s3_cache_sync.ts (2026.08.12.1), the pullChanged walk:
const stat = await Deno.stat(localPath);
if (stat.size === entry.size) {
// File exists locally with matching size — no download needed.
...
continue;
}while the push side (fileNeedsPush) fingerprints size → mtime → sha256 and
writes a fresh sha256 into the shared index on every upload. The asymmetry
means a writer correctly publishes a same-size update (new bytes, new sha256
in the index) and every reader already holding an old same-size copy skips
the download on every future sync. The pull walk even reconciles the index
entry's localMtime to the reader's own file, suppressing the drift signal.
Repro (two clones, A and B, sharing one bucket):
A: write a synced file with content AAAA; datastore sync # push
B: datastore sync # B has AAAA
A: change the content to BBBB (same length); datastore sync
B: datastore syncExpected: B reads BBBB. Actual: B reads AAAA, and no number of syncs repairs it — B stays stale until the file's size happens to change.
Upstream repository: https://github.com/systeminit/swamp-extensions
Environment
- Extension:
@swamp/s3-datastore@2026.08.12.1 - swamp:
20260820.072114.0-sha.05d20b6a - OS:
linux(x86_64) - Deno:
2.8.3 - Shell:
/bin/bash
Closed
No activity in this phase yet.
stack72 commented 8/28/2026, 5:07:23 PM
Fixed in PR #239. pullChanged now compares sha256 (with an mtime fast path) when sizes match, mirroring the push side's existing size→mtime→sha256 chain. The index already carried the sha256 — pull just wasn't reading it.
Sign in to post a ripple.