Skip to main content
← Back to list
01Issue
BugShippedSwamp CLI
Assigneesstack72

Relationships

#1052 commitPush no-op verification fetches the full monolithic index instead of the v2 shard/commitSeq fast-path (follow-up to #1034)

Opened by mgreten · 7/9/2026· Shipped 7/9/2026

Thanks again for the shard-first work and the #1034/#1049 fixes — the index and catalog paths are both fast now. This is a small follow-up in the same family: one code path in commitPush still fetches the full monolithic index instead of using the v2 shard/commitSeq fast-path, so a no-op model method run on a v2 repo pays a multi-second monolith GET that shouldn't be needed.

What we observe

On a fully migrated v2 repo, a model method run that pushes nothing traces:

[s3-sync] pullChanged.fastpath 306ms hit      # v2 fast path — great
[s3-sync] preparePush.fastpath 184ms hit      # v2 fast path — great
[s3-sync] commitPush 37992ms noop             # ← 38s on a no-op

pullChanged and preparePush both fast-path in ~200-300ms, but commitPush spends ~38s doing nothing.

What we traced

commitPush's pushed === 0 && deleted === 0 branch (s3_cache_sync.ts:2114) calls:

const indexFingerprint = await this.pullIndex({ forceRemote: true, signal });
if (indexFingerprint && this.index && !isMultipartETag(...) &&
    await this.localHasAllRemoteEntries()) {
  await this.markSynced(indexFingerprint);
}

pullIndex({ forceRemote: true }) does a direct getObject(this.indexKey()) of the monolithic .datastore-index.json — it doesn't attempt shard assembly the way pullChanged/pushChanged do (they call assembleIndexFromShards / pullPartitionedIndex first and only fall back to the monolith). So this verification path bypasses v2 entirely.

For us that object is 13.9MB; a raw GET of it measures ~23s over our ~1.1MiB/s uplink, which is the bulk of the 38s (the rest is localHasAllRemoteEntries stat-walking ~23k entries). Meanwhile the v2 truth is right there and tiny: _index/_meta.json is 13KB (with commitSeq), backed by 265 shards. The same commit was already validated by pullChanged's fast-path ~300ms earlier in the same run.

Ask (gently — your call on approach)

Could the commitPush no-op verification use the v2 commitSeq/_meta.json check (the same mechanism tryFastPullChanged already uses) when shard-first meta is present, and only fall back to the monolith pullIndex(forceRemote) for genuinely monolithic repos? On a v2 repo the 13KB meta carries everything this branch needs, so the 13.9MB GET looks like pure overhead. (If the branch's sidecar-sync bookkeeping is entirely opportunistic, another option might be to skip it on v2 when the earlier fast-path already marked things clean — but you'll know the invariants there far better than we do.)

Very low severity for us — our hot-path writers are on a local FS datastore, so this only bites explicit S3 model method runs (e.g. our daily adw-harvest), and the raised SWAMP_S3_REQUEST_TIMEOUT_MS keeps it from timing out. Just flagging it as the last monolith-sized GET we can still see on a v2 repo. Happy to share full traces and test any candidate build against our two-machine MinIO-over-Tailscale setup.

Environment

  • Extension: @swamp/s3-datastore@2026.07.09.1 (shard-first v2, migrated 2026-07-08)
  • swamp: 20260708.012813.0-sha.0f7166fe
  • OS: darwin (aarch64)
  • Setup: MinIO over Tailscale (~95ms RTT, ~1.1MiB/s uplink from the Mac)
  • Related: follow-up to #1034 (index shard-first) — same "v2 path not taken" shape, in the commitPush no-op branch

Upstream repository: https://github.com/systeminit/swamp-extensions

Environment

  • Extension: @swamp/s3-datastore@2026.07.09.1
  • swamp: 20260708.012813.0-sha.0f7166fe
  • OS: darwin (aarch64)
  • Deno: 2.8.3
  • Shell: /bin/zsh
02Bog Flow
✓OPEN✓TRIAGED✓IN PROGRESS✓SHIPPED+ 1 MOREASSIGNED+ 5 MOREREVIEW+ 4 MOREPR_MERGED+ 1 MORECONTRIBUTOR_NOTIFIED

Shipped

7/9/2026, 8:44:54 PM

Click a lifecycle step above to view its details.

03Sludge Pulse
stack72 assigned stack727/9/2026, 5:50:02 PM
Editable. Press Enter to edit.

stack72 commented 7/9/2026, 8:45:07 PM

Thanks @mgreten for reporting this! The fix has been merged and a release is on its way. We appreciate your contribution to swamp.

mgreten commented 7/10/2026, 10:22:05 PM

Confirmed working — thank you! Updated the fleet to 2026.07.09.2 and verified against a true no-op model method run on our v2 repo:

[s3-sync] commitPush 191ms v2noop      # was ~38s (13.9MB monolith GET)

The new v2noop path reads _meta.json first and uses the commitSeq instead of the monolith GET, exactly as hoped — and it still runs localHasAllRemoteEntries so the #1225 cache-completeness invariant is preserved. Nice.

End-to-end effect: a no-op adw-harvest (our daily S3 writer) dropped from ~137s to ~13s wall — the residual is just our Ruby run-dir scan, not the datastore anymore. That was the last monolith-sized GET we could see on a v2 repo, so from our side the shard-first migration is now fully paying off across pull, prepare, commit, and catalog. Thanks for the quick turnaround on all four of these (#1034/#1049/#1052 + the migration) — the shared S3 datastore went from multi-minute stalls to sub-second on the hot paths. Glad to test anything else against our setup whenever useful.

Sign in to post a ripple.