Skip to main content
← Back to list
01Issue
BugShippedSwamp CLIPublicTeam
Assigneesstack72

Relationships

#1795 Workflow summary omits failed assertion steps

Opened by jeremy · 8/23/2026· Shipped 8/23/2026

Problem

@swamp/workflow-summary can contradict the durable workflow result when a non-model task fails.

A workflow containing one successful model method followed by a failed assertion correctly records the workflow and job as failed. The generated summary omits the assertion, reports zero failed steps, and labels the job succeeded.

This makes the default diagnostic report hide the task that actually caused the workflow failure.

Environment

Swamp:

20260822.235828.0-sha.4046cd5c

Workflow:

  • Name: workflow-summary-failed-assertion
  • ID: 97881a97-03c0-4649-b03d-72a02c40cd34
  • Run: 11665cff-4064-47c1-b681-8d95df979fae

Minimal reproduction

Use any successful model method followed by a deliberately failing assertion:

jobs:
  - name: reproduce
    steps:
      - name: successful-model-step
        task:
          type: model_method
          modelType: "@swamp/git"
          modelName: weir-source-git
          methodName: status
          inputs: {}

      - name: deliberately-failed-assertion
        dependsOn:
          - step: successful-model-step
            condition:
              type: succeeded
        task:
          type: assert
          expr: "false"
          message: Deliberate assertion failure for workflow-summary reproduction.
          severity: high

Run the workflow and inspect its generated @swamp/workflow-summary.

Durable workflow result

The workflow run correctly records:

  • workflow status: failed
  • job reproduce: failed
  • step successful-model-step: succeeded
  • step deliberately-failed-assertion: failed
  • failedStep: deliberately-failed-assertion
  • failure reason: Deliberate assertion failure for workflow-summary reproduction.

Actual summary

# workflow-summary-failed-assertion: failed

1 succeeded · 0 failed · 0 skipped

## Job: reproduce (succeeded)

| Step | Model | Status |
| ---- | ----- | ------ |
| successful-model-step | weir-source-git → status | succeeded |

The JSON summary similarly reports:

{
  "status": "failed",
  "totalSteps": 1,
  "succeeded": 1,
  "failed": 0,
  "failures": []
}

Expected behavior

The summary should include all workflow task outcomes, not only model-method executions.

For this run it should:

  • count two total steps;
  • count the assertion as failed;
  • identify deliberately-failed-assertion and its error;
  • label job reproduce as failed.

Source-supported diagnosis

WorkflowReportContext.stepExecutions is model-shaped: each entry requires model name, model type, method name, model ID, method arguments, and data handles.

execution_service.ts::runWorkflowReports constructs this collection by iterating modelInfoByStep. Non-model tasks such as assertions therefore never enter the report context.

workflow_summary_report.ts then derives all counts, failures, per-job grouping/status, Markdown, and JSON solely from that incomplete collection.

The existing workflow-summary tests construct only model-shaped step records and do not cover a failed assertion or another non-model task.

Suggested repair boundary

Provide workflow reports with the complete set of task-step outcomes, including task type and error/assertion details. Keep model-specific metadata and retrieval commands optional for model-method steps.

Derive summary counts and per-job status from that complete task set.

Regression coverage should verify:

  1. a failed assertion is counted and shown with the job marked failed;
  2. an ordinary successful model-method workflow retains its current summary behavior.
02Bog Flow
✓OPEN✓TRIAGED✓IN PROGRESS✓SHIPPED+ 1 MOREASSIGNED+ 2 MOREREVIEW+ 6 MOREPR_MERGED+ 2 MORESESSION_SUMMARIZED

Shipped

8/23/2026, 8:29:24 PM

Click a lifecycle step above to view its details.

03Sludge Pulse
stack72 assigned stack728/23/2026, 7:08:57 PM
Editable. Press Enter to edit.

stack72 commented 8/23/2026, 8:29:42 PM

Thanks @jeremy for reporting this! The fix has been merged and a release is on its way. We appreciate your contribution to swamp.

Sign in to post a ripple.