Skip to main content
← Back to list
01Issue
FeatureShippedUAT
Assigneesstack72

Relationships

#1099 UAT Phase 2: run history and run doctor

Opened by stack72 · 7/11/2026· Shipped 7/11/2026

Summary

Phase 2 (run family) adds the first dedicated tests for swamp run history and swamp run doctor. These commands query and maintain the per-repo SQLite run tracker (.swamp/run_tracker.db). No coverage exists today — the commands are referenced in adversarial tests and model cancel tests but never directly tested.

New directory tests/cli/run/ with 2 test files. Lands in the existing misc CI shard (the Phase 8b split will eventually move it to families, but for now misc covers it — verify with the CI guard test).

Dependency

Phase 8a landed. No other dependencies.

Existing State

  • No tests/cli/run/ directory exists — needs creating
  • runWithSignal helper exists for SIGKILL injection (needed for stale-run fixture)
  • The shell-blocking-stdin fixture (tests/cli/fixtures/models/shell-blocking-stdin.yaml) provides a long-running process for the stale-run setup
  • model/cancel_test.ts already starts and cancels runs but does NOT test run history or run doctor directly
  • The misc CI shard includes root-level and multi-family paths — verify tests/cli/run/ is covered (it should be, since misc includes stragglers)

CI shard check: The misc shard currently lists explicit paths. tests/cli/run/ is a NEW directory that may not be in the shard list. Check during implementation whether it needs adding to the shard paths in ci.yml and uat.yml — if so, add it (Phase 8b wiring). The CI guard test will fail if it's not wired.

Verified Contracts

run history

  • Data source: .swamp/run_tracker.db (SQLite). Requires an initialized repo.
  • --active + --all → "--active and --all are mutually exclusive", exit 1
  • Default window: last 24h or still-running
  • Log columns: STATUS KIND NAME ID AGE PID HOST (STATUS gets [STALE] suffix for stale runs)
  • Empty → "No tracked runs.", exit 0
  • JSON: {runs:[{id, runKind, modelType, methodName, workflowName, pid, hostname, status, startedAt, heartbeatAt, stale}]}
  • Requires a repo — running outside a repo → "Not a swamp repository: …", exit 1

run doctor

  • Stale = status "running" AND heartbeat older than 90 seconds (STALE_TTL_MS, active_run.ts:20)
  • --fix reaps stale runs (marks them failed) when: same-host + PID dead, or unconditionally for cross-host entries
  • Log footer: "Reaped N stale run(s)." or "Run with --fix to automatically reap stale runs."
  • JSON: {totalTracked, active, stale, reaped, activeRuns, staleRuns}
  • Clean repo (no stale runs) → exit 0, stale: 0

New Test Files

run/history_test.ts (NEW)

Tests for swamp run history [--active] [--all]:

  • --active + --all → exact mutual-exclusion error message, exit 1
  • After a model method run completes → run history shows a row with correct [REDACTED-SECRET]
  • Empty (fresh repo, no runs) → "No tracked runs.", exit 0
  • JSON schema validated: RunHistorySchema
  • Non-repo directory → "Not a swamp repository: …", exit 1
  • --active shows only running (empty when nothing is running)
  • --all shows beyond the 24h default window (if feasible to test — may need to manipulate the DB timestamp, or just verify the flag is accepted and returns valid JSON)
  • Exit codes: 0/1

run/doctor_test.ts (NEW)

Tests for swamp run doctor [--fix]:

  • Clean repo (no stale runs) → exit 0, JSON {stale: 0, reaped: 0}
  • Stale run fixture: SIGKILL a model method run mid-flight using runWithSignal, then make the heartbeat stale. Two approaches (verify in Task 0 which works):
    • Option A: Wait 90+ seconds (too slow for CI)
    • Option B: Directly manipulate the heartbeat_at column in .swamp/run_tracker.db via a SQLite write (preferred — instant, deterministic)
  • After creating a stale run: run doctor detects it (JSON stale: 1), log says "Run with --fix to automatically reap stale runs."
  • run doctor --fix reaps the stale run: second run doctor shows stale: 0; run history shows the run with status failed
  • JSON schema validated: RunDoctorSchema
  • Exit codes: 0

Stale Run Fixture Strategy

The recommended approach for creating a stale run:

// 1. Start a long-running method run
const proc = await runWithSignal(runner, repo.dir,
  ["model", "method", "run", "blocking-model", "execute"],
  "SIGKILL", { delayMs: 500 }); // Kill after 500ms

// 2. Wait for the process to actually die
await proc.waitForExit();

// 3. Manipulate the heartbeat_at in run_tracker.db to be >90s old
// Use Deno's SQLite or shell out to sqlite3
const dbPath = `${repo.dir}/.swamp/run_tracker.db`;
// UPDATE active_runs SET heartbeat_at = datetime('now', '-120 seconds') WHERE status = 'running';

// 4. Now `run doctor` will see it as stale

Task 0 must verify: (a) that runWithSignal with SIGKILL leaves a "running" row in the tracker (the process dies before cleanup), and (b) that the SQLite schema uses heartbeat_at as the column name. Adjust if the schema differs.

Schemas

Add to src/cli/helpers/schemas.ts:

  • RunHistorySchema — {runs:[{id, runKind, modelType, methodName, workflowName, pid, hostname, status, startedAt, heartbeatAt, stale}]}
  • RunDoctorSchema — {totalTracked, active, stale, reaped, activeRuns, staleRuns}

Use .passthrough() (matching existing codebase convention).

Ground Rules

  • Read CLAUDE.md in the repo root first
  • Tests are more accurate than the code under test. Assert the documented contract. If swamp disagrees, the test stays red and file upstream.
  • Use the github-pr skill for commits and PR creation
  • Task 0 — ground truth: Build tip-of-main swamp, run run history, run history --json, run doctor, run doctor --fix --json by hand. Verify the SQLite schema and column names. Check if there's a heartbeat override env var. Note any drift.
  • Check open swamp-uat PRs for in-flight work before starting

Test Pattern

import { assertEquals } from "@std/assert";
import {
  createRunnerFromEnv,
  installFixtureModel,
  runCommand,
  runJsonCommand,
  runWithSignal,
  withInitializedRepoContext,
} from "@swamp-uat/helpers";

const runner = createRunnerFromEnv();

Deno.test("swamp run history shows completed method runs", () =>
  withInitializedRepoContext(runner, async (repo) => {
    await installFixtureModel(runner, repo.dir, "shell-echo");
    await runCommand(runner, repo.dir, ["model", "method", "run", "shell-echo", "execute"]);
    const result = await runJsonCommand(runner, repo.dir, ["run", "history"], RunHistorySchema);
    assertEquals([HOST-1] > 0, true);
    assertEquals(result.runs[0].status, "completed");
  }));

Every test MUST:

  • Spawn the real swamp binary
  • Assert an explicit exit code
  • Validate --json output through a Zod schema

How to Run & Verify

# 1. Format, lint, type-check
deno task fmt
deno task lint
deno task check

# 2. Framework unit tests (CI guard — will fail if new dir isn't wired)
deno task test

# 3. Run the run tests directly
SWAMP_NO_TELEMETRY=1 SWAMP_SOURCE_DIR=~/code/swamp-club/swamp deno test --allow-read --allow-write --allow-env --allow-run --allow-net tests/cli/run/

# 4. Full CLI suite
SWAMP_NO_TELEMETRY=1 SWAMP_SOURCE_DIR=~/code/swamp-club/swamp deno task uat:cli

CI Shard Wiring

tests/cli/run/ is a NEW directory. It must be added to the misc shard paths in both ci.yml and uat.yml. The CI guard test will fail if this isn't done. Add tests/cli/run/ to the misc shard's paths string.

Definition of Done

  • tests/cli/run/ directory created with 2 test files (history_test.ts, doctor_test.ts)
  • CI shard wiring: tests/cli/run/ added to misc shard paths in ci.yml and uat.yml
  • Both Zod schemas added to src/cli/helpers/schemas.ts
  • Stale-run fixture working (SIGKILL + heartbeat manipulation)
  • Every test spawns the real swamp binary — no mocking
  • Every test asserts an explicit exit code
  • Every --json test validates through a Zod schema
  • deno task fmt && deno task lint && deno task check && deno task test all green (guard passes)
  • SWAMP_NO_TELEMETRY=1 SWAMP_SOURCE_DIR=~/code/swamp-club/swamp deno task uat:cli passes
  • PR opened via github-pr skill, branching from main
02Bog Flow
✓OPEN✓TRIAGED✓IN PROGRESS✓SHIPPED+ 1 MOREASSIGNED+ 5 MOREREVIEW+ 3 MOREPR_MERGED+ 1 MORENOTIFICATION_SKIPPED

Shipped

7/11/2026, 11:45:21 PM

Click a lifecycle step above to view its details.

03Sludge Pulse
stack72 assigned stack727/11/2026, 10:40:58 PM

Sign in to post a ripple.