Skip to main content
← Back to list
01Issue
FeatureShippedUAT
Assigneesstack72

Relationships

#1111 UAT Phase 2: telemetry, update --check, version/help

Opened by stack72 · 7/13/2026· Shipped 7/13/2026

Summary

Bundled Phase 2 issue covering three small, independent command families that share a trait: they're non-repo or repo-light commands with no external dependencies.

  • telemetry stats — 1 new file, new directory tests/cli/telemetry/
  • update --check — 1 new file in existing tests/cli/update/
  • version subcommand + help schema — 2 new files at tests/cli/ root level

Total: 4 new test files + smoke_test.ts consolidation.

Dependency

Phase 8a landed. No other dependencies.

Existing State

  • tests/cli/update/background_test.ts and setup_auto_test.ts exist — do NOT duplicate
  • tests/cli/smoke_test.ts exists with swamp --version and swamp --help (flag tests) — the coverage plan says to fold smoke_test.ts into version_test.ts + Phase 6 matrix
  • tests/cli/help_flags_test.ts exists with --help flag tests on specific commands — this file is Phase 6's to delete, leave it alone here
  • No tests/cli/telemetry/ directory exists — needs creating
  • No tests/cli/version_test.ts or tests/cli/help_schema_test.ts exist

Verified Contracts

telemetry stats

  • Requires an initialized repo
  • Reads .swamp/telemetry/telemetry-<date>-<uuid>.json files
  • --days default 2; non-numeric → Cliffy validation exit 1 (exit code 2 for CLI validation — verify in Task 0)
  • Zero/negative days yields empty range → "No telemetry data found." / {message:"No telemetry data found"}
  • Emits JSON in BOTH modes (log and --json)
  • Stats shape: {totalInvocations, successCount, errorCount, userErrorCount, successRate, errorRate, commandFrequency, optionFrequency, averageDurationByCommand, platformDistribution, daysAnalyzed}
  • Non-repo directory → "Not a swamp repository: …", exit 1
  • To generate data: run a few commands with telemetry ON (unset SWAMP_NO_TELEMETRY for those spawns only, or don't set it at all)

update --check

  • HEAD request to hard-coded https://artifacts.swamp-club.com/... with 5s AbortSignal.timeout — NO URL override
  • Dev builds short-circuit to up_to_date with NO network call (isDevBuild, update_service.ts:74-81)
  • A SWAMP_SOURCE_DIR binary IS a dev build → tests are offline-safe
  • Assert: {status:"up_to_date", currentVersion} and log "swamp is up to date (<v>)"
  • Release binaries: assert only the anti-hang bound + valid result shapes (out of scope for this phase since we test with dev builds)

version subcommand

  • swamp version log → swamp <VERSION> (version string prefixed with "swamp ")
  • swamp version --json → {version} (just the version field)
  • swamp -V (the flag) → prints the BARE version string and ignores --json — -V and version --json are NOT identical; pin both
  • These are NON-REPO commands — no withInitializedRepoContext needed

help (hidden subcommand)

  • swamp help (hidden) → {version, root:CliCommandSchema}
  • CliCommandSchema = {name, description, arguments:[{name, required, variadic}], options:[{flags, description, required, default?, collect}], subcommands:[…]} (builtin help/version options filtered out)
  • Subtree: swamp help model method → strips global options, returns only the model method subtree
  • Unknown path → stderr Unknown command: swamp <path>, exit 1
  • This schema test is the foundation for Phase 6's drift ratchet — build CliSchemaSchema in schemas.ts as a recursive Zod schema
  • NON-REPO command — no repo needed

New Test Files

telemetry/stats_test.ts (NEW)

  • No data (fresh repo) → "No telemetry data found." / {message:"No telemetry data found"}, exit 0
  • Populated: run a few commands with SWAMP_NO_TELEMETRY unset (or explicitly set to empty string), then telemetry stats --json → stats schema validated, commandFrequency contains the command that was run
  • --days with invalid value (non-numeric) → exit 1 (verify exit code in Task 0 — may be 1 or 2 for Cliffy validation)
  • JSON emitted in BOTH modes (verify shape even without --json)
  • Non-repo directory → "Not a swamp repository: …", exit 1
  • Exit codes: 0/1

update/check_test.ts (NEW)

  • Dev build (SWAMP_SOURCE_DIR binary): update --check --json → {status:"up_to_date", currentVersion}, exit 0
  • Dev build log mode: output contains "swamp is up to date"
  • Bounded runtime: the command completes within a reasonable time (no 5s network hang since dev builds skip the network)
  • JSON schema validated
  • Exit codes: 0

version_test.ts (NEW — top-level)

  • swamp version → stdout matches swamp <semver> pattern (regex)
  • swamp version --json → {version} schema validated, version string matches semver
  • swamp -V → bare version string (no "swamp " prefix), ignores --json (assert -V --json output is identical to -V alone, NOT JSON)
  • -V output ≡ the version field from version --json (same string)
  • Fold the existing smoke_test.ts --version test into this file, then delete smoke_test.ts (move the --help test to a comment or leave it for Phase 6 to absorb)
  • Exit codes: 0

Note on smoke_test.ts consolidation: The coverage plan says to fold smoke_test.ts into version_test.ts. The --version flag test moves here. The --help flag test can either move here too or be left for Phase 6's global flags matrix to absorb. Check during implementation which makes more sense — if moving it is trivial, do it; otherwise leave a comment.

help_schema_test.ts (NEW — top-level)

  • swamp help --json → full schema shape via CliSchemaSchema (recursive Zod schema)
  • Root schema has name: "swamp", subcommands array is non-empty, known commands (model, workflow, vault, data) appear in subcommands
  • Subtree: swamp help model method --json → schema rooted at "method", global options stripped
  • Unknown path: swamp help nonexistent --json → stderr Unknown command: swamp nonexistent, exit 1
  • The CliSchemaSchema is the recursive schema — it must handle arbitrary nesting of subcommands
  • Exit codes: 0 for valid paths, 1 for unknown paths

Schemas

Add to src/cli/helpers/schemas.ts:

  • TelemetryStatsSchema — full stats shape
  • UpdateCheckSchema — {status, currentVersion, ...}
  • VersionSchema — {version}
  • CliSchemaSchema — recursive: {name, description, arguments, options, subcommands: z.lazy(() => CliSchemaSchema).array()}

Use .passthrough() (matching existing codebase convention).

CI Shard Wiring

  • tests/cli/telemetry/ is a NEW directory — must be added to the misc shard paths in ci.yml and uat.yml
  • version_test.ts and help_schema_test.ts are top-level files — check if they're already covered by the misc shard (the misc shard lists specific root-level files). If not, add them.
  • If smoke_test.ts is deleted, remove it from the misc shard paths
  • The CI guard test will fail if wiring is wrong

Ground Rules

  • Read CLAUDE.md in the repo root first
  • Tests are more accurate than the code under test. Assert the documented contract. If swamp disagrees, file upstream.
  • Use the github-pr skill for commits and PR creation
  • Task 0 — ground truth: Build tip-of-main swamp, run each command by hand. Especially verify: telemetry stats exit code for invalid --days, swamp -V --json behavior, swamp help full output shape.
  • Check open swamp-uat PRs before starting

How to Run & Verify

deno task fmt
deno task lint
deno task check
deno task test

# Telemetry test needs telemetry ON for the "populated" test
SWAMP_SOURCE_DIR=~/code/swamp-club/swamp deno test --allow-read --allow-write --allow-env --allow-run --allow-net tests/cli/telemetry/

# Update and version/help tests
SWAMP_NO_TELEMETRY=1 SWAMP_SOURCE_DIR=~/code/swamp-club/swamp deno test --allow-read --allow-write --allow-env --allow-run --allow-net tests/cli/update/check_test.ts tests/cli/version_test.ts tests/cli/help_schema_test.ts

# Full CLI suite
SWAMP_NO_TELEMETRY=1 SWAMP_SOURCE_DIR=~/code/swamp-club/swamp deno task uat:cli

Definition of Done

  • 4 test files created: telemetry/stats_test.ts, update/check_test.ts, version_test.ts, help_schema_test.ts
  • smoke_test.ts consolidated into version_test.ts (or clear plan for Phase 6 to absorb remaining)
  • All Zod schemas added including recursive CliSchemaSchema
  • tests/cli/telemetry/ added to misc shard paths
  • version_test.ts and help_schema_test.ts wired into misc shard
  • Telemetry test generates data by running commands with telemetry enabled
  • Every test spawns the real swamp binary — no mocking
  • Every test asserts an explicit exit code
  • Every --json test validates through a Zod schema
  • deno task fmt && deno task lint && deno task check && deno task test all green (guard passes)
  • SWAMP_NO_TELEMETRY=1 SWAMP_SOURCE_DIR=~/code/swamp-club/swamp deno task uat:cli passes
  • PR opened via github-pr skill, branching from main
02Bog Flow
✓OPEN✓TRIAGED✓IN PROGRESS✓SHIPPED+ 1 MOREASSIGNED+ 2 MOREREVIEW+ 5 MOREPR_MERGED+ 1 MORENOTIFICATION_SKIPPED

Shipped

7/13/2026, 9:34:47 PM

Click a lifecycle step above to view its details.

03Sludge Pulse
stack72 assigned stack727/13/2026, 3:20:35 PM

Sign in to post a ripple.