Skip to main content
← Back to list
01Issue
BugOpenSwamp CLIPublic
AssigneesNone

Relationships

#4221 Flaky: v3 replica check stall test reads restore.pid before the stub writes it

Opened by stack72 · 10/31/2026

The integration test "v3 replica check: a restore that stalls past the deadline ends the check without a verdict (swamp-club#3935)" in integration/datastore_v3_replica_verify_test.ts (line 340) is flaky.

It failed in a verify-build run on an unrelated branch (access grant subjects, swamp-club#4220), one failure in 34110 tests:

error: NotFound: No such file or directory (os error 2): readfile .../swamp-verify-stall-/restore.pid at datastore_v3_replica_verify_test.ts:378

Locally it passes alone and fails under --repeats 5 (deno run test integration/datastore_v3_replica_verify_test.ts --filter /stalls past the deadline/ --repeats 5).

Cause (from reading the test): the stub litestream is a shell script that first writes its own pid to restore.pid, then execs sleep. verifyReplica is given a 300 ms deadline. When the machine is loaded, the deadline can fire and kill the stub before the shell has written restore.pid, so the test reads a file that never appeared. The test assumes the stub always writes its pid before the deadline.

Possible fix: poll for restore.pid with waitFor before reading it, and treat a stub killed before it wrote its pid as already gone, or give the stub a longer deadline and assert on the kill differently. Per AGENTS.md, keep it correct under deno test --repeats.

02Bog Flow
◉OPEN○TRIAGED○IN PROGRESS○SHIPPED

Open

10/31/2026, 5:31:16 AM

No activity in this phase yet.

03Sludge Pulse

Sign in to post a ripple.