Extend the existing workflow resume command (currently gate-suspended runs only) with a --from <step> flag that re-enters a failed workflow's DAG at a specified step. Combined with the guard field (#1437, shipped in #1993), this enables safe workflow recovery — guards decide which steps actually re-execute.

Motivation

When a workflow step fails, the user currently has two options: re-run the entire workflow from scratch, or abandon the run. There's no way to resume from the failed step. For workflows with expensive or irreversible earlier steps (instrument dispensing, API provisioning, long computations), re-running from scratch is wasteful or dangerous.

This was identified during the #1430 design as the workflow-level recovery mechanism. Guard handles idempotency (what should run), resume handles re-entry (where to start).

Design

The `--from` flag

$ swamp workflow resume <run> --from <step>

Re-enters the DAG at the named step. All steps are re-evaluated — guards decide what actually executes. Steps before and at the same level as --from that have truthy guards are skipped. The failed step (and anything downstream) re-runs.

Example flow

$ swamp workflow run plate-assay run-1
  ✓ step dispense-reagent              data committed
  ✓ step calibrate                     data committed
  ✗ step read-plate failed             USB timeout at well 60
  – step analyse (skipped, depends on read-plate)

$ swamp workflow resume run-1 --from read-plate
  – step dispense-reagent (guarded)    ← guard truthy, didn't dispense twice
  – step calibrate (guarded)           ← guard truthy, calibration reused
  ✓ step read-plate                    ← re-executed, all 96 wells written
  ✓ step analyse                       ← downstream runs normally

forEach handling

--from targets step template names as written in the workflow YAML, not expanded iteration names. For forEach steps, the platform re-expands all iterations and re-evaluates guards — completed iterations are skipped, failed/unstarted iterations execute.

- name: read-plate
  forEach:
    item: well
    in: ${{ range(1, 97) }}
  guard: ${{ data.latest("plate-reader", self.well) }}
  task:
    modelName: plate-reader
    method: read-single-well

$ swamp workflow resume run-1 --from read-plate
  – read-well-1 (guarded)
  ...
  – read-well-59 (guarded)
  ▸ read-well-60               ← failed last time, no data
  ✓ read-well-60
  ▸ read-well-61               ← never ran, no data
  ...
  ✓ read-well-96

Implementation notes

Three status gates in the current code restrict resume to suspended runs and need to be relaxed for failed runs when --from is provided:

resolveSuspendedRun() in suspended_run_resolver.ts — filters for status === "suspended"
WorkflowExecutionService.resume() at line ~2023 — checks existingRun.status !== "suspended"
CLI command's findWaitingApprovalStep() check at line ~270

The existing gate-suspended resume path must be preserved alongside the new --from path. The two should branch early in the CLI command.

Safety

Completed steps are not re-run — for safety, not just efficiency. Re-running a completed step can repeat an irreversible action: dispense liquid twice, re-send an API call, re-provision infrastructure. The guard is the mechanism that prevents this.

Steps without a guard always execute on resume. This is the correct default — if you didn't write a guard, you're saying "always run this step."

#1430 — parent issue (writeResource atomicity)
#1437 — guard field (shipped in #1993)
rollbackOnFailure — method-level deferred-latest write mode (separate issue, addresses the original #1430 writeResource concern)

02Bog Flow

Shipped

7/28/2026, 4:55:21 PM

Click a lifecycle step above to view its details.

03Sludge Pulse

stack72 assigned stack727/28/2026, 3:24:58 PM

model validate: transport-reachable preflight sends unresolved vault CEL expression as credential

Add `amp` (Amp Code) as a built-in --tool for repo init/upgrade/audit

Add rollbackOnFailure for method-level write atomicity

Expose resource attributes in DataHandle for workflow report stepExecutions

Make swamp serve durable and highly available

Docs: document guard and resume --from in workflows reference

.swamp.yaml rewritten with quote style change + lastSkillMigrationWarning on read-only operations

Add --from flag to workflow resume for DAG re-entry on failed runs

data delete removes data locally but push reports 'no changes' — next datastore sync restores everything

Empty report returns persist 0-byte versions that mask the real report — and `swamp data gc` then deletes the real version, keeping the empties

Docs: document guard field for workflow steps

First-class scheduled garbage collection policy per datastore namespace

Add guard field to workflow steps for idempotent execution

--quiet mode does not suppress in-process extension logger output

tailscale transport hangs indefinitely on Tailscale SSH check-mode auth; never surfaces the login URL

Concurrent @type auto-creation races: N simultaneous runs create N definitions sharing one name, and only one is reachable

Worker queries need a safe connected-worker filter for fleet fan-out

Extension docs still ship the Driver extension type, which the binary refuses (rejectRemovedDriverFields) after #535

Docs: document vault selection for sensitive fields (defaultVault, per-resource vaultName)

azure-kv error wrapping loses original error type — ERROR_TYPE always 'Error' in traces

writeResource commits per-call: a method that fails midway leaves its earlier outputs persisted, with no marker that the run failed

Docs: document workflow-scope ReportContext fields in extension report reference

Cloudflare: expose GraphQL Analytics API (httpRequestsAdaptiveGroups / client-IP request analytics)

aws-sm deleteAnnotation removes all non-aws: tags, not just swamp-specific ones

No way to select which vault receives a sensitive-marked field's value

aws-sm: namespace annotation labels with swamp:label: prefix

workflow assert: a step whose CEL expression fails to evaluate is omitted from --junit output, so the XML reports a clean pass

Documentation for workflow assert steps and JUnit output

Expose workflow inputs in WorkflowReportContext

Teach swamp skill how to write assert workflow steps

Include a workflow's declared `inputs` in `workflow get --json` (and a `hasInputs` hint in `workflow search --json`)

Docs: add how-to guide for running swamp serve/worker in daemon mode

worker daemon enable: relative/missing --cache-dir fails launchd EX_CONFIG while daemon status reports running

swamp doctor extensions: nondeterministic BundleBuildFailed for local extensions on an unchanged tree

Docs: document datastore directory relocation during setup

vault extensions: emit OTel spans for vault operations (AWS SM, Azure KV, 1Password)

Issue redactor corrupts reports: dotted code identifiers masked as hostnames, loopback and RFC 5737 documentation IPs masked, angle-bracket placeholders half-eaten

data prune resolves auto-definitions at the local path, not the datastore path — deletes live models' data (verified: destroyed a working server token and an active grant)

workflow/job-lifetime data is never reclaimed — expiry check requires an ownerDefinition.workflowId that step execution never sets

runModel breadth limit (MAX_INVOCATION_BREADTH=100) is never enforced — tracking object is never written back to the caller context; verified 150/150 calls succeeded

serve daemon enable is ungated — it reports success, then the account gate fires inside the detached daemon, which crash-loops invisibly

Purely local operations require a swamp-club account: local-token serve is gated while deprecated --auth-mode none is not, third-party OIDC is gated, and 'Local_Encryption' is paywalled by capitalization

A collective token stored in auth.json (rather than SWAMP_API_KEY) is permanently scope-less — resolution only runs for the env var, and scopes are never persisted for collective tokens

An empty stored apiKey counts as authenticated — saveIdentityCache writes apiKey:'' on a virgin machine, permanently satisfying the mandatory-account gate with no credential

Account/scope gates fire before argument, type and repo validation — 'swamp vault create env my-vault' (the command's own example, an invalid type) returns a paywall instead of 'Unknown vault type'

auth token create advertises a 'collective:write' scope that does not exist, and following its advice hits 'collective tokens cannot create other collective tokens'

requireScope does exact string matching — fine-grained scopes fail gates that demand the literal wildcard, and minted scopes are never validated or normalized

Collective-token scope cache never expires, is never refreshed by whoami, and cannot be cleared via the CLI — scope changes and revocations never take effect

Collective-token scope cache is keyed on a 12-char token prefix — only 2 chars distinguish org tokens, so a rotated token silently inherits another token's scopes

Misleading 'Your collective token lacks the X scope' when scope resolution failed — startup whoami errors are silently swallowed

Community points / reputation: award points for upvoting extensions, feature requests, comments, and feed posts

globalArgs fields leak into method arguments when a method's arguments schema is a bare z.record()

swamp serve --auth-mode oauth: crash during first-run admin resolution discards the OAuth token, forcing full device-flow re-registration

Docs: document extension runtime permissions and device I/O workaround in user manual

datastore setup extension drops 'namespace' from --config, making extensions that require it impossible to configure

Kiro CLI v3 permissions: no capability covers disclose_context (skill loading)

CLI: swamp invite <email> to drive the platform-invite API

api-key-scoping explanation doc contradicts shipped fine-grained scopes

Fine-grained scopes for swamp.club access tokens

End-to-end UAT for fine-grained collective token scopes (#960)

Docs: document swamp auth token create command for collective tokens

open method fails for password-auth hosts: SSHPASS env set but ssh call never wrapped in sshpass

docs: update swamp skills for setup extension --namespace

apps sync/lookup fail with 'partial() cannot be used on object schemas containing refinements'

Docs: update autoGc documentation to reflect write-time version pruning

model search/list doesn't enumerate auto-created definitions under .swamp/auto-definitions/

CLI command to create scoped collective tokens (phase 8 of #960)

docs: document auth gates for team features (datastores, vaults, serve)

`datastore setup filesystem` never transfers content when leaving a sync-based (remote) datastore — copies only the catalog and hardcodes `filesPulled: 0`

Workflow/CLI output uses red text for non-error/informational lines, contradicting standard error-color convention

Support end-user timezones on https://swamp-club.com/u/shelson/activity

Document doctor datastores --repair for namespace contamination cleanup

version-drift check (from #236) is a raw string-inequality, false-positives on multi-model manifests and metadata-only bumps

Pre-flight checks receive unresolved vault.get() expression text, not the resolved secret

Invite people who have never used swamp to a collective (email invite flow)

docs: update giga-swamp manual pages for setup extension --namespace

Docs: update extension push output examples to include channel and visibility

Collective management API for extension-based automation

Remote --server wss:// fails through an HTTP/2 TLS reverse proxy (WS client ALPN is h2-only)

uat: add namespace isolation and shard resilience test coverage