Skip to main content
← Back to list
01Issue
BugOpenSwamp CLIPublic
AssigneesNone

Relationships

#3902 v3 crash recovery's progress line breaks --json stderr: a failing command that recovered prints text before its JSON error (regression from #3363 / #3893)

Opened by stack72 · 10/25/2026

Description

swamp#3363 (fixing #3893) has a v3 checkout's crash recovery print system │ Recovering work an interrupted command left in this checkout… on stderr in both output modes, through createLockProgressWriter("system"), which checks only -q. That choice is recorded in design/enablers/datastores.md.

It conflicts with an existing rule in design/surfaces/auth-gate.md (~line 106): "JSON mode has no console log output, and a failing command's stderr must stay one JSON document, so the warning is held until the run ends." #3363 does not mention that rule.

So a --json command that recovers an interrupted writer and then fails writes a text line followed by its JSON error document to stderr, and the stderr no longer parses as JSON:

 system │ Recovering work an interrupted command left in this checkout…
{"error": "Another swamp process took over the writer lease ...", "code": "datastore_v3_writer_fenced", ...}

Found by swamp-uat

Run: https://github.com/swamp-club/swamp-uat/actions/runs/40149814398, swamp 20261025.010734.0-sha.02a2b20b (#3363's merge commit). The failure is the same on s3-v3 and gcs-v3:

  • tests/cli/datastore/v3_datastore_test.ts: "swamp takes over the v3 writer lease of a writer on another machine only after its TTL, and a writer stopped there is fenced when it resumes", step "a writer stopped on another machine"
error: SyntaxError: Unexpected token 's', " system │ R"... is not valid JSON
  const refusal = RefusalSchema.parse(JSON.parse(result.stderr));

Steps to reproduce

  1. Use a v3 datastore (S3 or GCS) with two checkouts, A and B.
  2. In B, start swamp model method run <slow-model> execute --json and SIGKILL it while it holds the writer lease. This leaves writer.json in B's checkout.
  3. In B, start swamp model method run <slow-model> execute --json again. It recovers first, which prints the line to stderr. Stop it (SIGSTOP) mid-body.
  4. In A, run a write and let it wait out the lease TTL and take over.
  5. Resume B. It is fenced and exits 1.
  6. B's stderr is the recovery line followed by the datastore_v3_writer_fenced JSON document, and JSON.parse(stderr) fails.

More generally: any --json command that runs an interrupted-writer recovery and then fails.

Expected

In --json mode, a failing command's stderr is one JSON document. Either the recovery progress line does not go to stderr in JSON mode, or it is held the way the auth-gate warning is. Log mode keeps the line, which is what #3893 wanted; -q still silences it; --json stdout is unchanged. Update datastores.md to say "in log mode".

The lock and sync progress lines use the same writer and also ignore the output mode, so they can break the same rule when a --json command waits on a lock and then fails. A fix at the writer level would cover both.

Impact

Low today: v3 datastores are not GA, so no user can reach this. It should be fixed before v3 GA, because agents and scripts driving swamp with --json against long-lived checkouts parse the error code from stderr. Exit codes and data are unaffected. Until then, the s3-v3 and gcs-v3 UAT shards fail on every release run.

Environment

  • swamp 20261025.010734.0-sha.02a2b20b (Latest release)
  • swamp-uat datastore matrix, s3-v3 and gcs-v3 backends, Linux CI runner
02Bog Flow
◉OPEN○TRIAGED○IN PROGRESS○SHIPPED

Open

10/25/2026, 1:54:56 AM

No activity in this phase yet.

03Sludge Pulse

Sign in to post a ripple.