Relationships
#3685 Flaky: datastore_v3_outbox_model_property_test throws MayLandLaterError on a random seed (aged commit, failed discard-and-mark checkpoint, power cut)
Opened by zack · 10/20/2026· Shipped 10/20/2026
What happens
The unseeded first case of integration/datastore_v3_outbox_model_property_test.ts (over random interleavings no commit lands after its failure, a takeover or off its head, no run names refused data, and no commit names a missing chunk) failed in verify-build for branch issue-3526 at commit a4fd943795dbca86a77083347c021a329fb32aa0. That branch changes worker stderr redaction only and does not touch datastore_v3; the other 27918 tests passed, and the same file passed on a rerun with a fresh seed. Found while verifying swamp-club#3526.
Reproduce
fast-check reported seed -1781746823, path 992:3:11:17:17:12, endOnFailure true, shrunk 5 times. Counterexample interleaving: [{kind: stage-refused}, {kind: checkpoint, fault: fail-append-and-discard}, {kind: age}, {kind: checkpoint, fault: fail-append-discard-and-mark}, {kind: restart, takeover: false}, {kind: power-cut, keep: 0}] Pass that seed and path to the first fc.assert (test line 837) to replay it. The second case in the file uses a fixed seed (3490), so only the unseeded case can surface this.
Error
MayLandLaterError thrown from throwUnconfirmed (src/infrastructure/datastore_v3/commit_log_unit_of_work.ts line 762) via finish and end, reached from runModel (test line 741): the commit of partition model--@swamp%2Faws-ec2--6f1c2a3b-4d5e-4f60-8a7b-9c0d1e2f3a4b has been retried for longer than 7 days; it was dropped. Swamp could not record that this change must not be committed (database or disk is full), so it may already be in the v3 datastore, or be committed by the next swamp command that writes it.
Expected
Either the model accepts MayLandLaterError for an aged entry whose discard-and-mark checkpoint failed before a power cut, or the unit of work handles that path without throwing. The test last changed in swamp-club#3586 (f3cfb9b53).
(First filing had the commit id and the replay path mangled by the CLI's redaction; see swamp-club#3686.)
Shipped
Click a lifecycle step above to view its details.
Sign in to post a ripple.