Relationships
#3241 Windows: workflow cancel and model cancel do not kill the step process tree
Opened by stack72 · 10/11/2026· Shipped 10/11/2026
workflow cancel and model cancel set the cancel flag but do not terminate running step processes. The product has terminateWindowsTree using taskkill /T /F in process_executor.ts but the cancel code path does not trigger it. Child Start-Sleep processes survive after cancel. Also causes serve shutdown to hang because the server child processes are not killed. Affects ~12 UAT tests across workflow cancel, model cancel, workflow run --timeout, and serve lifecycle. Discovered via Windows UAT runner swamp-uat#363.
Shipped
Click a lifecycle step above to view its details.
stack72 commented 10/11/2026, 2:03:47 PM
Verified on Windows Server 2022. When swamp workflow cancel is issued against a running workflow, the cancel flag is written to the database but the step process tree stays alive. Specific findings: (1) After workflow cancel --all, the step child process (Start-Sleep) remains running - assertNoProcessMatching finds PID still alive after 10s timeout. (2) model cancel shows same pattern - the model run command returns but child processes survive, causing os error 32 (file locked) on temp dir cleanup. (3) serve lifecycle tests hang because server child processes are not killed on SIGTERM shutdown. The product already has terminateWindowsTree in process_executor.ts (line 311-329) using taskkill /PID /T /F, and resolveProcessGroupIsolation returns false on Windows which should route to tree mode. The gap is between workflow cancel setting the database flag and the execution service actually calling terminateProcess with tree mode on the running step child. UAT tests affected: workflow cancel (4), model cancel (2), workflow run --timeout nested child (1), serve lifecycle drain (5+).
stack72 commented 10/11/2026, 3:46:29 PM
Tested with binary v20261011.153055.0-sha.8453c3cc which includes PR #2987. The sleep child process still survives after workflow cancel --all. The taskkill /T may not reach the Start-Sleep process because PowerShell spawns it outside the direct process tree visible to taskkill. The process probe found PID 6260 (sleep) still running 10s after cancel. Model cancel also still fails with the same binary.
stack72 commented 10/11/2026, 3:58:07 PM
Process tree kill is now working with the latest binary. 2 of the 4 workflow cancel test failures were test infra (process probe self-matching and sleep token pattern) which are now fixed. The remaining 2 failures are different: the step status on Windows is "cancelled: the process running this step stopped before the step finished" instead of just "cancelled". On POSIX a cancelled step records bare "cancelled" but on Windows the error includes the explanatory suffix. This is a step-error-recording difference, not a process kill issue.
stack72 commented 10/11/2026, 10:30:25 PM
Tested with latest binary 20261011.222452.0. Process tree kill works correctly. The remaining 2 test failures are about the step error message: on Windows a cancelled step records "cancelled: the process running this step stopped before the step finished" instead of bare "cancelled". On POSIX the step records just "cancelled" with no suffix. This error text difference causes assertEquals to fail at cancel_test.ts:572. The tests are: workflow cancel live local run and workflow cancel nested child.
Sign in to post a ripple.