Skip to main content
← Back to list
01Issue
FeatureShippedUAT
Assigneesstack72

Relationships

#1070 UAT Phase 1b: workflow family verb gap-fill

Opened by stack72 · 7/10/2026· Shipped 7/10/2026

Summary

Phase 1b fills coverage gaps in the workflow command family. The workflow family already has deep coverage for create, delete, validate, search, run (basic), schema get, and approvals (approve/reject/resume). This phase adds the missing verbs: cancel, edit, evaluate --all, get (dedicated), and history/run search with filters.

All new files land in the existing misc CI shard (which covers tests/cli/workflow/). Existing fixtures and helpers (installFixtureWorkflow, createWorkflow, runWorkflow, approveStep, etc.) are already available.

Dependency

Phase 8a (issue #1059) is landed.

Existing State (do NOT duplicate)

These workflow test files already exist:

  • tests/cli/workflow/schema/get_test.ts
  • tests/cli/workflow/search_test.ts
  • tests/cli/workflow/validate_test.ts

e2e tests with partial workflow coverage (do NOT duplicate these):

  • tests/cli/e2e/workflow_crud_test.ts — create/get/validate/delete lifecycle, nonexistent workflow errors
  • tests/cli/e2e/workflow_inputs_test.ts — input wiring, k=v, JSON, @file, dot-notation, coercion
  • tests/cli/e2e/cel_evaluate_test.ts — CEL evaluation (17 tests)
  • tests/cli/e2e/extension_workflow_test.ts — extension workflow runs

The workflow/history/ directory needs creating.

Useful existing fixtures:

  • tests/cli/fixtures/workflows/single-step.yaml — simplest workflow
  • tests/cli/fixtures/workflows/multi-step-pipeline.yaml — multi-step
  • tests/cli/fixtures/workflows/manual-approval-gate.yaml — long-running (for cancel tests)
  • tests/cli/fixtures/workflows/workflow-with-inputs.yaml — has inputs

Useful helpers: installFixtureWorkflow, createWorkflow, runWorkflow, approveStep, rejectStep, resumeWorkflow, listApprovals

New Test Files

workflow/cancel_test.ts (NEW)

Tests for swamp workflow cancel [workflow] --run <id> --all --reason:

  • Cancel an in-flight workflow run (use manual-approval-gate fixture — it blocks at the approval step, giving time to cancel) → run YAML status shows cancelled, visible in workflow history search
  • --run <id> targeting a specific run
  • --all with nothing active → {"cancelled":[]} / "No active workflow runs found to cancel.", exit 0
  • --all JSON shape: {cancelled:[{runId,workflowName,previousStatus}], count, reason}
  • Already-terminal run → "Run <id> is already in a terminal state (status: <s>)", exit 1
  • Nonexistent workflow → "Workflow not found: <x>", exit 1
  • Workflow exists but no active runs → 'No active runs found for workflow "<name>"', exit 1
  • --reason custom reason echoed in JSON output (default reason is "Cancelled by user")
  • No arg and no --all → UserError, exit 1
  • --json single cancel shape: {runId, workflowName, previousStatus, status:"cancelled", reason}
  • Exit codes: 0 on success, 1 on errors

workflow/edit_test.ts (NEW)

Tests for swamp workflow edit [workflow]:

  • No-arg + --json → UserError "Workflow ID or name is required in non-interactive mode", exit 1
  • Stdin-mode edit: pipe valid YAML on stdin → definition updated, JSON status:"updated"
  • Invalid stdin YAML → rejected with error message, exit 1
  • Nonexistent workflow → exit 1
  • NOT subject to the sensitive-arg chokepoint (unlike model edit — this is a meaningful difference to verify: piping a workflow YAML that references a vault expression in a global arg should succeed)

workflow/evaluate_test.ts (NEW)

Tests for swamp workflow evaluate [workflow] --all:

  • --all (or no arg) evaluates every workflow: writes to .swamp/workflows-evaluated/, log says "Evaluated {n} of {total} workflow definitions"
  • --all JSON shape: {items:[{id,name,hadExpressions,forEachExpanded?,outputPath?,jobs?}], total, evaluated}
  • Single workflow evaluate with --input (some e2e coverage exists — only add the --all + JSON-shape gaps that aren't covered)
  • Nonexistent workflow → "Workflow not found:", exit 1
  • Verify evaluated files actually land in .swamp/workflows-evaluated/ (stat check)
  • Exit codes: 0 on success, 1 on errors

workflow/get_test.ts (NEW)

Tests for swamp workflow get <workflow>:

  • Happy path: get a workflow → JSON schema validated
  • Nonexistent workflow → exit 1
  • Note: the e2e crud test has a "get fails for nonexistent" test, but it's inside a lifecycle journey — this file provides the dedicated command-dir coverage with proper JSON schema validation

workflow/history/search_test.ts (NEW)

Tests for swamp workflow history search AND swamp workflow run search:

  • Key contract: both commands query the SAME underlying data (all runs, sorted desc) with identical item shapes {runId, workflowId, workflowName, status, startedAt?, completedAt?, duration?, tags?}
  • Difference: run search supports --since --status --workflow --tag --limit(50); history search takes only [query], no filters/limit
  • Tests:
    • history search piped → auto-JSON with results array
    • history search with query string → filtered results
    • history search empty → empty results array, exit 0
    • run search --status completed → only completed runs
    • run search --tag K=V → only tagged runs
    • run search --workflow <name> → only that workflow's runs
    • run search --since 1h → time-bounded
    • run search --limit 1 → single result
    • Identical item shapes between both commands (same schema)
    • Both auto-JSON when piped (non-TTY), per §4.2 interactivity rules
    • Hidden list alias parity for workflow history
    • Exit codes: always 0 (search never fails on empty results)

Schemas

Add these new Zod schemas to src/cli/helpers/schemas.ts:

  • WorkflowCancelSchema (single cancel)
  • WorkflowCancelAllSchema (--all cancel)
  • WorkflowEditSchema
  • WorkflowEvaluateSchema (single)
  • WorkflowEvaluateAllSchema (--all)
  • WorkflowGetSchema
  • WorkflowRunSearchSchema / WorkflowHistorySearchSchema (may share an item schema since the shapes are identical)

Use .passthrough() on schemas (matching existing codebase convention).

Ground Rules

  • Read CLAUDE.md in the repo root first — its rules apply everywhere
  • Tests are more accurate than the code under test. Assert the documented contract. If swamp disagrees, the test stays red and you file the discrepancy upstream with swamp issue bug and reference it in a test comment. Never weaken a test to match current behavior.
  • Use the github-pr skill for commits and PR creation
  • Task 0 — ground truth: Before writing any test, build tip-of-main swamp and run each in-scope command by hand in a scratch repo (happy path, --json, one failure). Read the command definitions in ~/code/swamp-club/swamp/src/cli/commands/. Note any drift from this issue in the PR description.
  • Check open swamp-uat PRs for in-flight work touching your files before starting
  • Use grep -rl "workflow cancel\|workflow edit\|workflow evaluate\|workflow get\|workflow.*history" tests/cli/ before writing to avoid duplicating existing coverage

Test Pattern

All tests must follow this pattern — real binary, real commands, real exit codes:

import { assertEquals } from "@std/assert";
import { waitFor } from "cli-testing-library";
import {
  createRunnerFromEnv,
  installFixtureWorkflow,
  runCommand,
  runJsonCommand,
  runWorkflow,
  withInitializedRepoContext,
} from "@swamp-uat/helpers";

const runner = createRunnerFromEnv();

Deno.test("swamp workflow cancel cancels an in-flight workflow run", () =>
  withInitializedRepoContext(runner, async (repo) => {
    // 1. Arrange: install fixture workflow + model, start a run that blocks
    // 2. Act: run `swamp workflow cancel <name>`
    // 3. Assert: exit code, stdout/stderr content, JSON schema validation
    // 4. Verify post-state: history search shows cancelled status
  }));

Every test MUST:

  • Spawn the real swamp binary (via runCommand, runJsonCommand, or runner.render)
  • Assert an explicit exit code
  • Validate --json output through a Zod schema
  • End with waitFor(() => assertEquals(instance.hasExit()?.exitCode, N)) when using runner.render

How to Run & Verify

# 1. Format, lint, type-check
deno task fmt
deno task lint
deno task check

# 2. Run framework unit tests (includes the CI guard test — will fail if new files aren't wired)
deno task test

# 3. Run the workflow tests directly
SWAMP_NO_TELEMETRY=1 SWAMP_SOURCE_DIR=~/code/swamp-club/swamp deno test --allow-read --allow-write --allow-env --allow-run --allow-net tests/cli/workflow/

# 4. Run the full CLI suite to check for regressions
SWAMP_NO_TELEMETRY=1 SWAMP_SOURCE_DIR=~/code/swamp-club/swamp deno task uat:cli

CI Shard Wiring

All new files go under tests/cli/workflow/ which is already covered by the misc shard. The new tests/cli/workflow/history/ subdirectory is automatically included since the shard path is tests/cli/workflow/. No CI changes needed.

However: check that the Phase 8a guard test still passes after adding workflow/history/search_test.ts — the guard should already account for this since the misc shard covers tests/cli/workflow/ (trailing slash = all subdirectories).

Definition of Done

  • All 5 test files created per the table above
  • All Zod schemas added to src/cli/helpers/schemas.ts
  • Every test spawns the real swamp binary — no mocking, no importing swamp internals
  • Every test asserts an explicit exit code
  • Every --json test validates through a Zod schema
  • Upstream discrepancies filed with swamp issue bug and referenced in test comments
  • deno task fmt && deno task lint && deno task check && deno task test all green
  • SWAMP_NO_TELEMETRY=1 SWAMP_SOURCE_DIR=~/code/swamp-club/swamp deno task uat:cli passes
  • PR opened via github-pr skill, branching from main
02Bog Flow
✓OPEN✓TRIAGED✓IN PROGRESS✓SHIPPED+ 1 MOREASSIGNED+ 5 MOREREVIEW+ 3 MOREPR_MERGED+ 1 MORENOTIFICATION_SKIPPED

Shipped

7/10/2026, 8:40:00 PM

Click a lifecycle step above to view its details.

03Sludge Pulse
stack72 assigned stack727/10/2026, 4:44:57 PM

Sign in to post a ripple.