Relationships
#4802 serve shutdown: a nested child run cut off by shutdown ends cancelled without its cleanup, and the parent's resume starts a new child
Opened by zack · 11/16/2026· Shipped 11/16/2026
What happens
Follow-up to swamp-club#4763, where a serve shutdown interrupts runs instead of cancelling them, so no cleanup (always, completed or failed) step or job starts and the resume continues the run.
A step of type workflow runs a child with the parent's abort signal, so the child also takes the interruption and starts no cleanup. But:
- the child ends cancelled through the engine's cancel path;
- shutdown never rewrites it: interruptShutdownRuns only marks the top-level runs the API registry, webhook service and scheduler report (design/primitives/serve.md: shutdown does not track nested child runs);
- recovering and resuming the parent re-runs its call-child step, which always calls childService.run and starts a new child.
So the original child stays cancelled for good, and its cleanup never runs. Before #4763 the child ran its cleanup during shutdown (inside the draining serve) and then ended cancelled.
Reproduction
parent: job outer, step call-child, task type workflow, workflowIdOrName e2e. e2e: job main with step slow (sleep 120) and step cleanup (dependsOn slow, condition always); job notify (dependsOn main, condition always). Start the parent through the API, SIGTERM serve while slow runs. Records: parent interrupted (call-child unknown); child cancelled, slow failed (stoppedByAbort), cleanup and notify failed/cancelled (settledByAbort). Recover and resume the parent: a new child runs slow, cleanup and notify; the old child stays cancelled.
Expected
A child cut off by shutdown is interrupted like its parent, and the parent's resume continues that child rather than starting a new one, so the child's cleanup runs if the resumed child fails.
Likely shape
- interruptShutdownRuns (or the engine, on a RunInterruption) marks the interrupted parent's child runs interrupted, found through the parent steps' nested run links, inside the same write fence.
- On resume, a workflow step whose recorded nested run is interrupted continues that child instead of calling childService.run anew (nested_run_link.ts already distinguishes an interrupted child).
- Orphaned-nested-run handling must not cancel such a child before the parent resumes.
Shipped
Click a lifecycle step above to view its details.
Sign in to post a ripple.