A nested structural swamp still waits on its run's lock when the lock holder is not a same-host ancestor (cross-host worker, --server into a non-ancestor serve)

h12h ago

#3093

Deliver workflow signals through a write-once outcome record instead of writing the run record

h13h ago

#2982

Forward held-lock identity over --server so a loopback nested swamp does not wait on its caller's lock

h14h ago

#2983

A nested structural swamp on a same-host remote worker waits on its own step's lock held by swamp serve

h15h ago

#2981

Fail fast when nested structural commands under parallel runs wait on each other's locks

h15h ago

#3079

Docs: remove followUpActions from the MethodResult reference in the extension model manual

h15h ago

#3092

signal_test: already-settled test fails by chance when a random UUID spells the sender name

16h ago

#3068

Add a wait_for_signal workflow step that pauses a run for a JSON message

h16h ago

#3059

run doctor through serve interrupts a live run of another serve instance when no heartbeats are recorded

h17h ago

#3081

Flaky test: WalSink replayed segments are delivered before events written after replay leaves one WAL segment

h17h ago

#3078

Remove followUpActions: no model or extension can produce them

h18h ago

#2946

Workflow schedule watcher and workflow edit symlink lookup ignore the managed config workflows dir

h18h ago

#3084

main is red: repository_dirty_coverage_test fails for UnifiedData.collectGarbage(orphaned deferred write)

18h ago

#3082

isProcessGone always reports alive on Linux under the test task permissions, failing 9 tests on main

h18h ago

#2893

serve: relative tls cert-file/key-file and --config resolve against the working directory, not the repository

h18h ago

#2975

findBySpec/findByTag lose a workflow step's output after a version is deleted, pruned or rolled back

s18h ago

#2917

Run tracker keeps some interrupted workflow rows forever: retention waits for markSettled, which several paths never call

20h ago

#3051

run doctor reads every run record on each run to rebuild the run indexes

hs20h ago

#3037

Serve audit: two sinks with the same name share one emitter cursor, so the second never receives events

1d ago

#3039

Serve audit: a durable sink write that times out is retried while the first call is still pending, so the store can hold duplicate sequences

s1d ago

#3043

Flaky test: ModelResolver data accessors for a renamed model return the earlier id's record from findBySpec

s1d ago

#2985

data query --limit returns fewer live records than exist and reports limited: false when stale catalog rows fall inside the limit

s1d ago

#2987

data get: help example passes --run latest, which is not a recognised run id

s1d ago

#2905

workflow reject leaves sibling gates waiting_approval and dependents pending in a failed, retryable run

h1d ago

#2920

workflow cancel cannot cancel a suspended run whose workflow file was deleted, and cancel --all silently skips it

h1d ago

#2898

A job alone in its level starts after the abort, and its unstarted step is recorded as a real failure instead of settledByAbort

h1d ago

#2919

workflow approve racing workflow cancel loses the cancel: the run returns to suspended after cancel reported cancelled

h1d ago

#2972

data query: does not follow data rename forwarding that data get follows

h1d ago

#2994

Unit tests share fixed /tmp catalog paths, so concurrent runs corrupt them and fail every later run

h1d ago

#2968

data query: renamed data items are not found by their old name

h4d ago

#2960

data query: model lookup by id and orphan recovery parity with data get

h4d ago

#2899

workflow approve on a run that is not suspended prints a fatal error with a stack trace

h4d ago

#2957

A nested structural swamp spawned by swamp serve skips every lock serve holds, including other runs' locks

h4d ago

#2962

data query: confirm catalog parity with data get for custom datastores

h4d ago

#2961

data query: single-result --json mode for scripts migrating from data get

h4d ago

#2948

data get --workflow silently returns an arbitrary step's data when several steps share a data name

h4d ago

#2942

swamp doctor workflows misses nested and .yml extension workflows that the loader reads

h4d ago

#2928

auth whoami says the stored API key is no longer valid when auth.json only holds an env-key identity cache

s4d ago

#2921

Docs: swamp skill says cancelling a parent run cancels its nested child runs; the child is left suspended

s4d ago

#2941

Namespace advice in the slow-lock warning is wrong for a single repo on a local datastore

h4d ago

#2930

Method-run records left running by a dead owner are never reaped

s5d ago

#2909

Flaky: usecase_sync_characterization_data_test 'data gc (serve)' asserts markDirty order that varies under parallel load

s5d ago

#2926

Worker dispatch rebuilds the definition with Definition.create, so a legacy-named model cannot run on a worker

h5d ago

#2918

A cancelled workflow exits before its steps stop, leaving their method-run records at running

h5d ago

#2927

Nested swamp structural command in a shell step times out on its parent's per-model lock (SWAMP_LOCK_HOLDER_PID is stripped)

5d ago

#2924

Auth gate blocks nested swamp in workflow shell steps when the credential is in SWAMP_API_KEY

h5d ago

← Back to list

01Issue

BugShippedSwamp CLIPublic

Assigneeshammz

Relationships

#2920 workflow cancel cannot cancel a suspended run whose workflow file was deleted, and cancel --all silently skips it

Opened by hammz · 10/1/2026· Shipped 10/5/2026

PR #2840: https://github.com/swamp-club/swamp/pull/2840

Description

If a workflow's YAML file is deleted while one of its runs is suspended, that run cannot be cancelled from the local CLI:

swamp workflow cancel <wf> fails with "Workflow not found";
swamp workflow cancel <wf> --run <id> fails the same way;
swamp workflow cancel --all silently skips the run.

The run stays suspended indefinitely. The only way to cancel it is through swamp serve, which settles it correctly. Either POST /api/v1/cancel/workflow-run/<id> or swamp workflow cancel --run <id> --server <url> works.

Steps to reproduce

swamp workflow run gated --input env=x --json. The run suspends at a manual_approval gate; call its id $R.
rm workflows/workflow-<gated id>.yaml.
swamp workflow cancel gated --json exits 1 with Workflow not found: gated.
swamp workflow cancel gated --run $R --json exits 1 with Workflow not found: gated.
swamp workflow cancel --all --json exits 0 with {"cancelled":[],"finished":[],"deleted":[],"skipped":[]}. $R is still suspended.

Expected

workflow cancel --run <id> cancels a suspended run by its id, even when the definition has been removed. The run's own evaluated snapshot is still at .swamp/workflows-evaluated/runs/<id>/evaluated-workflow.yaml. Since swamp-club#2895, settlement can use that snapshot, or the run's records alone when there is no snapshot. Serve already cancels it this way.
--all includes such runs, or at least reports them under skipped with the reason.

Notes

This is the local-CLI counterpart of the "A suspended run whose workflow file was deleted cannot be cancelled" gap listed in the nested approval gates UAT plan. Reproduced with release 20261001.190402.0-sha.543aad7e and with the swamp-club#2895 branch (8ae02e19).

Environment: Linux x86_64, filesystem datastore.

02Bog Flow

Shipped

10/5/2026, 7:03:53 PM

Click a lifecycle step above to view its details.

03Sludge Pulse

hammz assigned hammz10/5/2026, 3:59:07 PM

A nested structural swamp still waits on its run's lock when the lock holder is not a same-host ancestor (cross-host worker, --server into a non-ancestor serve)

Deliver workflow signals through a write-once outcome record instead of writing the run record

Forward held-lock identity over --server so a loopback nested swamp does not wait on its caller's lock

A nested structural swamp on a same-host remote worker waits on its own step's lock held by swamp serve

Fail fast when nested structural commands under parallel runs wait on each other's locks

Docs: remove followUpActions from the MethodResult reference in the extension model manual

signal_test: already-settled test fails by chance when a random UUID spells the sender name

Add a wait_for_signal workflow step that pauses a run for a JSON message

run doctor through serve interrupts a live run of another serve instance when no heartbeats are recorded

Flaky test: WalSink replayed segments are delivered before events written after replay leaves one WAL segment

Remove followUpActions: no model or extension can produce them

Workflow schedule watcher and workflow edit symlink lookup ignore the managed config workflows dir

main is red: repository_dirty_coverage_test fails for UnifiedData.collectGarbage(orphaned deferred write)

isProcessGone always reports alive on Linux under the test task permissions, failing 9 tests on main

serve: relative tls cert-file/key-file and --config resolve against the working directory, not the repository

findBySpec/findByTag lose a workflow step's output after a version is deleted, pruned or rolled back

Run tracker keeps some interrupted workflow rows forever: retention waits for markSettled, which several paths never call

run doctor reads every run record on each run to rebuild the run indexes

Serve audit: two sinks with the same name share one emitter cursor, so the second never receives events

Serve audit: a durable sink write that times out is retried while the first call is still pending, so the store can hold duplicate sequences

Flaky test: ModelResolver data accessors for a renamed model return the earlier id's record from findBySpec

data query --limit returns fewer live records than exist and reports limited: false when stale catalog rows fall inside the limit

data get: help example passes --run latest, which is not a recognised run id

workflow reject leaves sibling gates waiting_approval and dependents pending in a failed, retryable run

workflow cancel cannot cancel a suspended run whose workflow file was deleted, and cancel --all silently skips it

A job alone in its level starts after the abort, and its unstarted step is recorded as a real failure instead of settledByAbort

workflow approve racing workflow cancel loses the cancel: the run returns to suspended after cancel reported cancelled

data query: does not follow data rename forwarding that data get follows

Unit tests share fixed /tmp catalog paths, so concurrent runs corrupt them and fail every later run

data query: renamed data items are not found by their old name

data query: model lookup by id and orphan recovery parity with data get

workflow approve on a run that is not suspended prints a fatal error with a stack trace

data query: no way to target a workflow's latest run

data query: binary content lacks contentEncoding/base64 parity with data get

data query: no fallback from spec name to data instance name

data query: no definitionHash or garbageCollection fields in query results

A nested structural swamp spawned by swamp serve skips every lock serve holds, including other runs' locks

data query: confirm catalog parity with data get for custom datastores

data query: single-result --json mode for scripts migrating from data get

data get --workflow silently returns an arbitrary step's data when several steps share a data name

swamp doctor workflows misses nested and .yml extension workflows that the loader reads

auth whoami says the stored API key is no longer valid when auth.json only holds an env-key identity cache

Docs: swamp skill says cancelling a parent run cancels its nested child runs; the child is left suspended

Namespace advice in the slow-lock warning is wrong for a single repo on a local datastore

Method-run records left running by a dead owner are never reaped

Flaky: usecase_sync_characterization_data_test 'data gc (serve)' asserts markDirty order that varies under parallel load

Worker dispatch rebuilds the definition with Definition.create, so a legacy-named model cannot run on a worker

A cancelled workflow exits before its steps stop, leaving their method-run records at running

Nested swamp structural command in a shell step times out on its parent's per-model lock (SWAMP_LOCK_HOLDER_PID is stripped)

Auth gate blocks nested swamp in workflow shell steps when the credential is in SWAMP_API_KEY

Relationships

#2920 workflow cancel cannot cancel a suspended run whose workflow file was deleted, and cancel --all silently skips it

Description

Steps to reproduce

Expected

Notes

Shipped