Skip to main content
← Back to list
01Issue
FeatureShippedUAT
Assigneesstack72

Relationships

#1068 UAT Phase 1a: model family verb gap-fill

Opened by stack72 · 7/10/2026· Shipped 7/10/2026

Summary

Phase 1a fills coverage gaps in the model command family. The model family already has deep coverage for get, search, validate, type describe/search, output get/search, and auto_resolve. This phase adds the missing verbs: cancel, delete, edit, output data, method history search, and method run advanced flags (--stdin, --tag, checks).

This is the cheapest type of coverage work — fixtures, helpers, and schemas already exist. All new files land in the existing model-vault-config-report CI shard.

Dependency

Phase 8a (CI guard + tier-6, issue #1059) must be landed first. It is now landed.

Existing State (do NOT duplicate)

These model test files already exist — do not recreate or duplicate their coverage:

  • tests/cli/model/get_test.ts
  • tests/cli/model/search_test.ts
  • tests/cli/model/validate_test.ts
  • tests/cli/model/auto_resolve_test.ts
  • tests/cli/model/type/describe_test.ts
  • tests/cli/model/type/search_test.ts
  • tests/cli/model/output/get_test.ts
  • tests/cli/model/output/search_test.ts
  • tests/cli/model/output/logs_test.ts (currently failure-only — extend with happy path)

These directories need creating: tests/cli/model/method/, tests/cli/model/method/history/

Useful existing fixtures:

  • tests/cli/fixtures/models/shell-echo.yaml — simple model that produces output
  • tests/cli/fixtures/models/shell-blocking-stdin.yaml — long-running model (for cancel tests)
  • tests/cli/fixtures/models/shell-producer.yaml — data-producing model
  • tests/cli/fixtures/models/shell-with-inputs.yaml — model with input arguments

New Test Files

model/cancel_test.ts (NEW)

Tests for swamp model cancel [model] --all --reason:

  • Start a long-running run using the shell-blocking-stdin fixture, cancel it → status is cancelled in run history
  • --all with nothing running → {"cancelled":[]} shape, exit 0
  • --all with log output → "No running model method runs to cancel."
  • No arg and no --all → UserError "Provide a model name or ID, or use --all…", exit 1
  • Nonexistent model → "Model '<x>' not found", exit 1
  • Model exists but no running runs → "No running method runs found for model '<name>'", exit 1
  • --json single cancel shape: {id, modelName, type, method, status:"cancelled", reason}
  • Assert the run-tracker view (run history) shows cancelled status, NOT the output YAML (output may be stale after SIGKILL)
  • Exit codes: 0 on success, 1 on errors

model/delete_test.ts (NEW)

Tests for swamp model delete <model> -f:

  • Happy path: delete a model → definition + outputs + data gone; model get fails after
  • JSON success shape: {deleted:{id,name,type,inputPath}, resourceDeleted, outputsDeleted, evaluatedInputDeleted, dataDeleted}
  • Model with data but no --force → blocked with "Model '<name>' has N associated data artifact(s). Delete the data first, or use --force to delete all."
  • Model referenced by a workflow → blocked with "…is referenced by workflow(s): <list>…"
  • Prompt decline → {"cancelled":true} / "Deletion cancelled."
  • Nonexistent model → exit 1
  • Exit codes: 0 on success, 1 on errors

model/edit_test.ts (NEW)

Tests for swamp model edit [model]:

  • No-arg + --json → UserError "Model ID or name is required in non-interactive mode", exit 1
  • Stdin-mode edit: pipe valid YAML on stdin → definition updated, JSON status:"updated"
  • Invalid stdin YAML → "Invalid model YAML from stdin: <err>", exit 1
  • Stdin edit swapping a sensitive arg to a literal → rejected with code literal_sensitive_global_arg
  • Nonexistent model → exit 1

model/output/data_test.ts (NEW)

Tests for swamp model output data <id> --field --version --name:

  • Content after a data-producing run
  • --field extraction on a JSON data object + error shapes: "Cannot extract field…" on non-object, lists available fields on missing field
  • --name selection + available-list error when name doesn't exist
  • Invalid id (<3 hex chars) → "Invalid output ID format…"
  • Ambiguous prefix → lists matches
  • Not found → error, exit 1
  • --json shape: {outputId, methodName, dataId, dataName, version, contentType, field, data}

model/output/logs_test.ts (EXTEND existing)

  • ADD happy-path test (currently only has failure path)

model/method/history/search_test.ts (NEW)

Tests for swamp model method history search:

  • Piped (non-TTY) → auto-JSON {query, results:[{id, definitionId, modelName?, type, methodName, status, startedAt, durationMs?}]}
  • With query string → filtered results
  • Empty history → results:[], exit 0
  • Hidden list alias parity (stdout identical to search)

model/method/run_test.ts (NEW)

Tests for advanced swamp model method run flags not yet covered:

  • --stdin NDJSON batch: multiple objects (one per line), each produces a run; log shows "Running method <m> [i/N]"
  • --stdin + --input merge: stdin line deep-merged with --input values
  • --stdin + --input-file → rejected with "Cannot combine --stdin with --input-file."
  • Failing batch item → exits 1 and stops (remaining items not run)
  • --tag K=V → tag visible on data records via data search --tag K=V
  • Bad tag format → 'Invalid tag format: "<x>". Expected KEY=VALUE'
  • Checks fixture: requires a test fixture extension model type that declares a checks entry. Check runs on mutating method; --skip-check <name> skips a named check; --skip-checks skips all; definition-required checks are immune to skip flags; failure message starts "Pre-flight checks failed for \"<def>\" → <method>:"
  • Skip/report flags + --server → rejected with "... not supported with --server yet"

Note on checks fixture: The checks tests require a model TYPE (extension) that declares pre-flight checks. You will need to create a fixture extension for this. Look at the existing extension test infrastructure in src/cli/helpers/ — specifically writeExtension, writeKindExtension, and KIND_CONFIGS — to understand how to scaffold a test extension with a checks-enabled model type.

Schemas

Every --json test must validate through a Zod schema in src/cli/helpers/schemas.ts. Add these new schemas:

  • ModelCancelSchema / ModelCancelAllSchema
  • ModelDeleteSchema
  • ModelEditSchema
  • ModelOutputDataSchema
  • ModelMethodHistorySearchSchema

Use .passthrough() on schemas (matching the existing convention in the codebase — the .strict() vs .passthrough() decision is deferred to Phase 6).

Ground Rules

  • Read CLAUDE.md in the repo root first — its rules apply everywhere
  • Tests are more accurate than the code under test. Assert the documented contract. If swamp disagrees, the test stays red and you file the discrepancy upstream with swamp issue bug and reference it in a test comment. Never weaken a test to match current behavior.
  • Use the github-pr skill for commits and PR creation
  • Task 0 — ground truth: Before writing any test, build tip-of-main swamp and run each in-scope command by hand in a scratch repo (happy path, --json, one failure). Read the command definitions in ~/code/swamp-club/swamp/src/cli/commands/. Note any drift from this issue in the PR description.
  • Check open swamp-uat PRs for in-flight work touching your files before starting
  • Use grep -rl "swamp model cancel\|swamp model delete\|swamp model edit" tests/cli/ before writing to avoid duplicating existing coverage

Test Pattern

All tests must follow this pattern — real binary, real commands, real exit codes:

import { assertEquals } from "@std/assert";
import { waitFor } from "cli-testing-library";
import {
  createRunnerFromEnv,
  installFixtureModel,
  runCommand,
  runJsonCommand,
  withInitializedRepoContext,
} from "@swamp-uat/helpers";

const runner = createRunnerFromEnv();

Deno.test("swamp model cancel cancels a running method run", () =>
  withInitializedRepoContext(runner, async (repo) => {
    // 1. Arrange: install fixture, start a long-running run
    // 2. Act: run `swamp model cancel <name>`
    // 3. Assert: exit code, stdout/stderr content, JSON schema validation
    // 4. Verify post-state: `run history` shows cancelled status
  }));

Every test MUST:

  • Spawn the real swamp binary (via runCommand, runJsonCommand, or runner.render)
  • Assert an explicit exit code
  • Validate --json output through a Zod schema
  • End with waitFor(() => assertEquals(instance.hasExit()?.exitCode, N)) when using runner.render

How to Run & Verify

# 1. Format, lint, type-check
deno task fmt
deno task lint
deno task check

# 2. Run framework unit tests (includes the CI guard test)
deno task test

# 3. Run the model shard tests
SWAMP_NO_TELEMETRY=1 SWAMP_SOURCE_DIR=~/code/swamp-club/swamp deno test --allow-read --allow-write --allow-env --allow-run --allow-net tests/cli/model/

# 4. Run the full CLI suite to check for regressions
SWAMP_NO_TELEMETRY=1 SWAMP_SOURCE_DIR=~/code/swamp-club/swamp deno task uat:cli

Definition of Done

  • All 7 test files created/extended per the table above
  • All Zod schemas added to src/cli/helpers/schemas.ts
  • Every test spawns the real swamp binary — no mocking, no importing swamp internals
  • Every test asserts an explicit exit code
  • Every --json test validates through a Zod schema
  • Upstream discrepancies filed with swamp issue bug and referenced in test comments
  • deno task fmt && deno task lint && deno task check && deno task test all green
  • SWAMP_NO_TELEMETRY=1 SWAMP_SOURCE_DIR=~/code/swamp-club/swamp deno task uat:cli passes (model shard green)
  • PR opened via github-pr skill, branching from main
02Bog Flow
✓OPEN✓TRIAGED✓IN PROGRESS✓SHIPPED+ 1 MOREASSIGNED+ 4 MOREREVIEW+ 4 MOREPR_MERGED+ 1 MORENOTIFICATION_SKIPPED

Shipped

7/10/2026, 4:40:59 PM

Click a lifecycle step above to view its details.

03Sludge Pulse
stack72 assigned stack727/10/2026, 2:46:26 PM

Sign in to post a ripple.