Relationships
#3193 Datastore sync: a no-op push after a model-scoped pull arms the remote commitSeq, hiding other hosts' commits until the next commit
Opened by stack72 · 10/8/2026
Summary
A repository whose datastore sync state is clean can record the remote's latest commitSeq after it pulled only one model's index shard. Every later pull then takes the commitSeq fast path and pulls nothing, so data other hosts committed stays missing locally until the next remote commit. Seen with @swamp/s3-datastore 2026.10.08.1 under full hydration; @swamp/gcs-datastore 2026.10.07.1 has the same code. Found while verifying swamp-club#3179; not related to that fix, and it reproduces on the released binary.
Repro
- Repos
controlandwriterboth runrepo init,extension pull @swamp/s3-datastoreanddatastore setup extension @swamp/s3-datastore ... --hydration-strategy fullon the same bucket prefix. Both hold model definitionsvpc(a command/shell model printing a value) andconsumer(whose run argument reads the vpc resource attribute through the model map). controlrunsswamp datastore sync. Its sidecar becomes commitSeq 0, localDirty false. (Any clean sidecar works; one synced at a later seq reproduces the same way.)writerrunsmodel method run vpc execute, thenmodel method run consumer execute. The remote commitSeq is now 2.controlrunsmodel method run consumer execute. It fails withModel vpc has no resource data, and the sidecar now records commitSeq 2.controlrunsswamp datastore sync:0 pulled, 0 pushed.model evaluate --allpulls nothing, and consumer keeps failing until some host commits again.
Cause (traced)
s3_cache_sync.ts, commitPush, the v2 no-op branch for a commit that pushed and deleted nothing (around line 3903-3921): it sets the sidecar's commitSeq to the remote meta's commitSeq when !this.indexIsPartial && this.index && localHasAllRemoteEntries(). Traced values in step 4: remote seq 2, inherited 0, indexIsPartial false, localHasAllRemoteEntries true, and this.index holding only consumer's 8 entries.
Every term passes because:
- The model-scoped
pullChanged(lock acquisition) merges only consumer's partition intothis.indexand correctly arms nothing itself. preparePushtakes its fast path:tryFastPushChangedreturns 0 for a clean sidecar, so it returns an empty manifest before assembling any shards andindexIsPartialkeeps its default false.commitPushcheckslocalHasAllRemoteEntries()against that model-scoped index, which is true, and arms the latest remote seq.
The GCS extension's gcs_cache_sync.ts has the same tryFastPushChanged, the same early empty-manifest return in preparePush, the same scoped partition merge and the same no-op arm in commitPush (source comparison only).
Suggested fix direction
Arming the latest remote commitSeq is only safe after a full index read (a full pull or a full shard assembly). When preparePush returns its fast-path empty manifest, the no-op branch of commitPush should keep the sidecar's existing commitSeq rather than write the remote's. Also mark this.indexIsPartial after a model-scoped partition merge in pullChanged, so no later arm in the same process trusts that index. Apply to S3 and GCS, with a conformance case: a scoped pull followed by a no-op push must not arm commitSeq past commits the cache never read.
Related
swamp-club#2418 (second comment) sees the same family from the other side: sidecar commitSeq not matching what is local, there leaving the fast path never armed.
Open
No activity in this phase yet.
system commented 10/8/2026, 8:29:50 PM
Classified automatically when this issue was filed.
- Source: Extensions
If you feel this classification is incorrect, add a ripple to tell us so.
Sign in to post a ripple.