Skip to main content
← Back to list
01Issue
BugShippedSwamp CLIPublic
Assigneeshammz

Relationships

#2548 workflow cancel on a live local run overwrites the owning process's final run record with a stale snapshot

Opened by hammz · 9/25/2026· Shipped 9/28/2026

Summary

The offline path of swamp workflow cancel (src/cli/commands/workflow_cancel.ts) loads the run record, then cancelRun() calls killProcessTree(run.pid), which sends SIGTERM and waits for the owning process to exit, then calls run.cancel(reason) on the record it loaded before the kill and saves it.

A local swamp workflow run process handles SIGTERM as an abort: it runs cleanup mode (swamp-club#1785), marks in-flight steps failed (cancelled), runs always/completed cleanup steps, and saves its final record. The cancel command then saves its stale snapshot over that record, so the persisted run loses every step outcome recorded after the snapshot was read, including cleanup steps that actually ran.

Steps to reproduce

  1. A workflow with a forEach step build over [1, 2, 3], concurrency 1, each iteration running sleep 5, and a step rollback with dependsOn build, condition type failed.
  2. Start swamp workflow run in the background, wait about 2.5s, then run swamp workflow cancel with a reason.
  3. The owning process's live output shows the cleanup-mode evaluation (rollback skipped (dependency), job failed, workflow cancelled).
  4. swamp workflow history get shows the persisted record instead: run cancelled with the cancel_reason tag, but job main pending, and steps build (the unexpanded forEach template name, not build-1..3) and rollback both pending, with no error or skipReason.

Observed with 20260925.180024.0-sha.e08f1d5b. The record is the snapshot saved at run start, before forEach expansion; the owning process had not saved again before the cancel command read it.

Expected

An offline cancel of a run whose owning process is alive and exits after SIGTERM keeps the owning process's final record: either re-read the run after the kill and cancel only if it is still non-terminal (adding the cancel_reason tag), or skip the write when the reloaded run is already terminal. A run whose owner was SIGKILLed or never wrote a terminal state still gets the cancelled status.

Context

Found while reproducing swamp-club#2543 (never-started dependencies stay pending after a cancel or --timeout). Once #2543 marks never-started steps terminal, this overwrite would still discard those marks when the cancel comes from another process.

02Bog Flow
✓OPEN✓TRIAGED✓IN PROGRESS✓SHIPPED+ 1 MOREASSIGNED+ 2 MOREREVIEW+ 14 MOREPR_MERGED+ 2 MORESESSION_SUMMARIZED

Shipped

9/28/2026, 4:20:33 PM

Click a lifecycle step above to view its details.

03Sludge Pulse
hammz assigned hammz9/28/2026, 3:27:30 PM

Sign in to post a ripple.