Skip to main content
← Back to list
01Issue
FeatureOpenSwamp CLIPublic
AssigneesNone

Relationships

#3483 Workflow run cancellation on concurrency-key conflict

Opened by swamp_lord · 10/14/2026

Problem

There's no way to configure a swamp workflow so that triggering a new run automatically cancels an existing in-flight run of "the same thing" — and have the old run's teardown happen in parallel with the new run's startup.

Concrete case: a CI-style workflow provisions worker VMs for a commit. A second push triggers a new run for an updated commit while the first is still provisioning/running. Today both runs execute independently with no relationship between them. The desired behavior:

  • The first run receives a cancel signal and moves into its teardown path (tear down the workers it created).
  • The second run is scheduled and starts provisioning its own workers immediately — not blocked on the first run's teardown finishing. Teardown and new-run startup proceed fully in parallel, no ordering between them.

What already exists (so this is additive, not new plumbing)

  • Cancellation itself is mature: swamp workflow cancel already runs cancelAndSettle (src/domain/workflows/abort_settlement.ts), which settles in-flight jobs/steps as cancelled and marks the run cancelled.
  • "Post-Cancellation Cleanup" already runs dependsOn: { condition: failed | completed | always } steps (src/domain/workflows/trigger_condition.ts) with a grace period even during cancellation — this is already the "tear down the workers" flow; there's no separate onFail field needed.
  • cancelAndSettle is not awaited by anything that would block a new run, so "parallel teardown + new-run startup" falls out for free once something actually triggers the cancel on the old run.

The actual gap

There is no concept of a cross-run concurrency/dedup key, and no mechanism that detects "a second run of this workflow (for the same thing) is already running" in order to trigger the cancel automatically. The closest existing mechanism, supersedeSuspendedRuns (src/libswamp/workflows/supersede.ts), only cancels suspended runs with exact input deep-equality — it doesn't reach running runs, and it isn't keyed by anything other than full input equality.

Proposed shape (for discussion, not a spec)

  • A new concurrency.key field on the workflow schema (src/domain/workflows/workflow.ts) — a template/CEL expression resolved against the run's inputs at trigger time into a string "concurrency key" / variant identifier. Workflow authors decide what makes two runs "the same thing" (e.g. branch, PR number, an explicit label) — this is deliberately left open rather than hard-coded to "same workflow name" or "same inputs."
  • On workflow run, resolve the key and look up any other run of this workflow in pending/running status with the same resolved key. This needs the active-run tracking in src/serve/active_run_registry.ts generalized from its current global/per-principal caps to an index keyed by (workflowId, concurrencyKey).
  • If a match is found: fire cancelAndSettle against the existing run without awaiting it, then proceed to schedule the new run's own jobs immediately.

Explicitly open questions (deferring to issue discussion, not blocking filing)

  1. Default concurrency-key scope when a workflow doesn't configure one — should every run of the workflow be one cohort by default, or should there be no implicit dedup without an explicit key?
  2. Should the superseded run land in the existing cancelled terminal status (tagged with a reason, e.g. supersededBy: <runId>), or does this warrant a distinct terminal status separate from a manually/externally cancelled run? Both are representable via the existing cancelAndSettle path either way, so this doesn't block the rest of the design.
02Bog Flow
◉OPEN○TRIAGED○IN PROGRESS○SHIPPED

Open

10/14/2026, 9:26:05 PM

No activity in this phase yet.

03Sludge Pulse

Sign in to post a ripple.