Relationships
#3483 Workflow run cancellation on concurrency-key conflict
Opened by swamp_lord · 10/14/2026
Problem
There's no way to configure a swamp workflow so that triggering a new run automatically cancels an existing in-flight run of "the same thing" — and have the old run's teardown happen in parallel with the new run's startup.
Concrete case: a CI-style workflow provisions worker VMs for a commit. A second push triggers a new run for an updated commit while the first is still provisioning/running. Today both runs execute independently with no relationship between them. The desired behavior:
- The first run receives a cancel signal and moves into its teardown path (tear down the workers it created).
- The second run is scheduled and starts provisioning its own workers immediately — not blocked on the first run's teardown finishing. Teardown and new-run startup proceed fully in parallel, no ordering between them.
What already exists (so this is additive, not new plumbing)
- Cancellation itself is mature:
swamp workflow cancelalready runscancelAndSettle(src/domain/workflows/abort_settlement.ts), which settles in-flight jobs/steps ascancelledand marks the runcancelled. - "Post-Cancellation Cleanup" already runs
dependsOn: { condition: failed | completed | always }steps (src/domain/workflows/trigger_condition.ts) with a grace period even during cancellation — this is already the "tear down the workers" flow; there's no separate onFail field needed. cancelAndSettleis not awaited by anything that would block a new run, so "parallel teardown + new-run startup" falls out for free once something actually triggers the cancel on the old run.
The actual gap
There is no concept of a cross-run concurrency/dedup key, and no mechanism
that detects "a second run of this workflow (for the same thing) is already
running" in order to trigger the cancel automatically. The closest existing
mechanism, supersedeSuspendedRuns
(src/libswamp/workflows/supersede.ts), only cancels suspended runs with
exact input deep-equality — it doesn't reach running runs, and it isn't
keyed by anything other than full input equality.
Proposed shape (for discussion, not a spec)
- A new
concurrency.keyfield on the workflow schema (src/domain/workflows/workflow.ts) — a template/CEL expression resolved against the run's inputs at trigger time into a string "concurrency key" / variant identifier. Workflow authors decide what makes two runs "the same thing" (e.g. branch, PR number, an explicit label) — this is deliberately left open rather than hard-coded to "same workflow name" or "same inputs." - On
workflow run, resolve the key and look up any other run of this workflow inpending/runningstatus with the same resolved key. This needs the active-run tracking in src/serve/active_run_registry.ts generalized from its current global/per-principal caps to an index keyed by (workflowId, concurrencyKey). - If a match is found: fire
cancelAndSettleagainst the existing run without awaiting it, then proceed to schedule the new run's own jobs immediately.
Explicitly open questions (deferring to issue discussion, not blocking filing)
- Default concurrency-key scope when a workflow doesn't configure one — should every run of the workflow be one cohort by default, or should there be no implicit dedup without an explicit key?
- Should the superseded run land in the existing
cancelledterminal status (tagged with a reason, e.g.supersededBy: <runId>), or does this warrant a distinct terminal status separate from a manually/externally cancelled run? Both are representable via the existingcancelAndSettlepath either way, so this doesn't block the rest of the design.
Open
No activity in this phase yet.
Sign in to post a ripple.