Skip to main content
← Back to list
01Issue
BugOpenSwamp CLIPublic
AssigneesNone

Relationships

#4188 Read-model catch-up of many data versions slows sharply under load

Opened by skunk-ape · 10/30/2026

Found while triaging swamp-club#4182.

A v3 read-model catch-up that applies 7,000 data versions (one commit store, appended directly, then a serve request triggers catchUp) takes about 715 ms on an idle host, and about 15 ms for 70 versions, so roughly linear at about 0.1 ms per version.

Under verify-build's full parallel test suite the same catch-up took over 90 seconds: the integration test 'a behavior whose floor is past the small reply limit still takes its lease through the client (swamp-club#4166)' in integration/serve_datastore_lease_fence_test.ts ran 1m36s and aborted, while its sibling tests in the same file took 0.4 to 1 s (about 5x their idle time). The catch-up slows by about 100x, far more than other work under the same load, which suggests per-version I/O (a file write, and possibly a sync, for each materialised metadata.yaml) dominates and degrades under disk contention.

Not a regression and not new behaviour; filed so the cost of a large first catch-up (a serve or checkout coming up on a partition with thousands of data versions) can be measured and judged on its own. Evidence: verify-build runs 7baf0224 and 777ac5f6 logs on 2026-10-30; the per-phase timings above came from an instrumented copy of the #4166 test.

Suggested first step: time catch-up of 7,000 versions under synthetic disk load and break it down by phase (CatchUpStats already records phases).

02Bog Flow
◉OPEN○TRIAGED○IN PROGRESS○SHIPPED

Open

10/30/2026, 5:38:07 PM

No activity in this phase yet.

03Sludge Pulse

Sign in to post a ripple.