Relationships
#3183 serve never continues a signalled run after a manual resume re-entered its open wait
Opened by zack · 10/8/2026· Shipped 10/8/2026
Found by the adversarial review of swamp-club#3108 (PR 2932). Not fixed there.
What happens
- A run is suspended on a
wait_for_signalstep that has no outcome yet. - Someone runs
swamp workflow resumeon it. The resume takes the continuation claim for the suspension as a local holder, saves the run as running, re-enters the wait, finds it still open, and suspends the run again. - The claim is kept, because the resume saved something.
- The signal is then accepted, through serve or through a local
swamp workflow signal. - Serve tries to continue the run, at once and then on every sweep. It finds the suspension claimed by a local command and refuses. After a minute it logs and audits
held_by_local_command, which says the command died.
The run stays suspended until someone runs swamp workflow resume a second time. A workflow with auto-resume on is not resumed by its signal.
Why
suspensionKeyOf (src/domain/workflows/continuation_claim.ts) derives the key from the run record, and its comment says a resume always changes at least one step. That is not true here: a step waiting on an open signal wait keeps its status, its start time and its wait id through the resume, so the second suspension has the same key as the first, and the local claim on that key still stands. The liveness of a local holder is always unknown, so an automatic resume never replaces it.
The same applies across serve instances on a synced datastore: after a manual resume through instance A, peer B treats A's claim on the unchanged key as held for as long as A is alive.
Possible fixes
- When a resume ends with the run suspended again, compare the new suspension key with the claimed one and give the claim back if they are equal.
- Or add something to the key that changes on every resume, such as a counter on the run record bumped in
resumeFromSuspended.
Test to add
Resume a run whose wait is still open, then settle the wait, and check that continueSettledRun launches the resume.
Shipped
Click a lifecycle step above to view its details.
Sign in to post a ripple.