Relationships
↔ sibling #3673#3645 v3 outbox model property test fails: a commit dropped after a failed append-discard-and-mark reports MayLandLaterError instead of the expected outcome
Opened by zack · 10/19/2026· Shipped 10/20/2026
Problem
integration/datastore_v3_outbox_model_property_test.ts, test "v3 outbox (model): over random interleavings no commit lands after its failure, a takeover or off its head, no run names refused data, and no commit names a missing chunk", fails for some random seeds. Seen in a verify-build run on an unrelated branch (swamp-club#3633), with fast-check seed -1721839444.
Shrunk counterexample (7 shrinks):
[{"kind":"stage","target":0,"big":false},
{"kind":"checkpoint","fault":"fail-append-and-discard"},
{"kind":"age"},
{"kind":"checkpoint","fault":"fail-append-discard-and-mark"},
{"kind":"restart","takeover":false}]The run ends with:
MayLandLaterError: The commit of partition definition--aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa has been retried for longer than 7 days; it was dropped. Swamp could not record that this change must not be committed (database or disk is full), so it may already be in the v3 datastore, or be committed by the next swamp command that writes it.
at throwUnconfirmed (src/infrastructure/datastore_v3/commit_log_unit_of_work.ts:762)
at finish (commit_log_unit_of_work.ts:779)
at end (commit_log_unit_of_work.ts:753)
at runModel (integration/datastore_v3_outbox_model_property_test.ts:741)Reproduce
Replace the test's fc.assert options with numRuns: 1 and examples set to the counterexample above. It fails the same way on origin/main at 0a2856a9f and on 22349c50b, so it predates the branch where it was seen.
Expected
Either the model accepts this outcome for an aged entry whose drop could not be recorded (the error says it may land later), or the outbox handles the sequence so the invariant holds. Whichever is right, the property should pass for every seed; a seed-dependent failure blocks unrelated verifications.
Notes
- Not the same as swamp-club#3640, which is a timeout in the killed-writer test under CPU load.
- Probably related to the discard and mark paths changed in swamp-club#3586 and swamp-club#3506.
Shipped
Click a lifecycle step above to view its details.
Sign in to post a ripple.