Lab #3675: Docs: doctor datastores reference for the v3 content check and lost format marker repair

What changed (swamp-club#3617)

swamp doctor datastores gained two user-facing behaviours for v3 datastores that the manual's doctor reference (content/manual/reference/doctor.md) doesn't describe. The page still says --repair is v2-only.

A v3_content health check. It reports:
- unreadable records (whose chunk is missing or damaged in the datastore);
- checkout files held aside in _read_model_v3/at-risk/;
- chunks stored at the wrong size.
Each entry comes with advice for its record kind. In JSON the check appears as v3ContentFinding (null when clean). A failing check exits 1, and so does a state that can't be inspected.
--repair [-y] restores a lost v3 format marker.
- When it applies: only after the format guard refuses a filesystem datastore configured with format 3 whose datastore-format.json is missing or blank beside its commits.db.
- Preview: without -y it only checks the files and exits 0.
- Restore: -y writes the marker.
- JSON statuses: format_marker_preview, format_marker_restored, format_marker_not_needed. Error code: format_marker_repair_failed.

Related user-visible error code: reads of an unreadable v3 record fail with datastore_v3_record_unreadable.

Pages to update

content/manual/reference/doctor.md: the datastores section, its JSON examples and the --repair description.
Possibly content/manual/how-to/troubleshoot-your-swamp-repo.md: a short "unreadable v3 record / lost format marker" entry.

The in-repo design reference is design/enablers/datastores.md (Format Marker, Loud reads, opt-in doctor sections) in swamp at the #3617 merge.

Inbox API contract and SwampClubClient inbox methods: summary, list, get and mark-read, pinned by a shared fixture and design/surfaces/inbox.md

Datastore rework Phase 5 (team mode) 2/3: the CLI's serve upstream — with serverAddress, a v3 checkout commits, reads chunks and catches up through serve's datastore API, with no storage credentials

Scheduled flaky-test detection is failing

v3 read model: guard against aliased key spellings on case-insensitive filesystems, without slowing replay

Scheduled flaky-test detection is failing

stagecraft remote studio: one long-lived connection to serve instead of a swamp process per read

stagecraft remote studio poll: follow-ups from #3658 review (flaky test, mid-poll first page, quiet-poll test)

docs(manual): the manual documents swamp subscription status and start — standings, call to action, flags and --json output

swamp subscription start opens the Swamp Club billing page, or a collective's with --collective, and status points its call to action at it

v3: the compaction policy bounds how much history a fresh clone replays (retainMs, minCommits, retainCommits) and the base load skips files the tail removes

datastore_v3_replica_verify_test stall case flakes under the full integration run: restore.pid is read before the stub writes it

v3 on S3/GCS: a replica restore yields a corrupt commits.db (fails SQLite's integrity check), so fresh clones and restores of the datastore fail; reproduces with Litestream alone from its first compacted L1 file (found by #3916)

swamp subscription status reports the plan and billing standing of the signed-in user and each collective, worded as auth whoami words it

/o/<slug>/billing sends a signed-out visitor to the collective page, or a 404 when private, instead of through login and back like /billing

swamp-extensions GCS storage test 'put: an abort mid-upload leaves no object and no resumable session' fails about 1 run in 5 (fixed 50 ms sleeps)

refactor(cli): share auth whoami's collective-standing helpers ahead of swamp subscription, and pin Personal License and Admin standings

@swamp/cve/tailworm: check the reachability matrix against a Tailscale-hosted tailnet

AWS codegen treats nested primaryIdentifier pointers as top-level fields

AWS codegen collapses nested createOnly/readOnly paths to their last segment

Generated state schemas require every child of nested objects, so normal resources warn on create/get

swamp-club.com/cve/tailworm states two things its sources do not say: finding 5's memory growth and no exploitation in the wild

cve/tailworm extension README says the findings were not seen in the wild, which no source states

Datastore rework Phase 5 (team mode) 1/3: serve's datastore API — commit, chunk and catch-up endpoints, authorisation on each op's intent, principal stamping and actionable conflicts

@swamp/cve/tailworm: reachability follow-ups from #3919 review

@swamp/cve/tailworm: opt-in reachability matrix from each node's packet filter

Datastore rework Phase 5: content-level authorisation for forwarded definition and evaluated-file writes

swamp-club.com/cve/tailworm: cover grade, probeScript, gradeFleet and the reachability matrix (extension 2026.10.25.5)

Docs: --trusted-hosts accepts host:port for public addresses on non-default ports

@swamp/cve/tailworm: after mitigate caps headscale, the rescan still tells you to set MemoryMax and Restart=always

@swamp/cve/tailworm grade: unix serve targets the probe could not resolve are graded as ordinary sockets

kubernetes: externalsecret and normalizeMeta follow-ups from #3907

v3 crash recovery's progress line breaks --json stderr: a failing command that recovered prints text before its JSON error (regression from #3363 / #3893)

Docs: vaults reference should say a wide local_encryption key file gets a warning

Docs: manual states the old pre-flight check rule and scores fast-check by deno doc --lint

Docs: define the sensitive-output fallback vault order and document the doctor vaults fallback warning

v3 long silent waits: crash recovery runs ~24 s with no output, data prune is silent 6-10 s in log mode, and a lagging read hangs ~112 s when Litestream's listing is cut (found by #3756)

Add @swamp/cve/http2shell: scan and mitigate the Brotli decoder RCE (CVE-2026-111497, Node2Shell/HTTP2Shell)

extension_tarball_analyzer_test fails on macOS: tar --mode is GNU-only

S3/GCS datastore: a push that fails after uploading drops its recorded deletes, so the retry brings deleted data back

v2 S3: about 4-5% of serial workflow runs fail with datastore_unpublished_data_version ('a concurrent run or the serve poller synced') when nothing else is running (found by #3756)

Datastore rework Phase 4 I: dogfood and benchmark solo v3 (filesystem, S3, GCS) in a throwaway repo with the released binary, check the kill criteria, and decide whether to announce

v3 bucket outage: a command cut off while becoming the writer says its changes are saved locally when nothing was saved (workflow approve), and the suggested recover fails for a run cut mid-step (found by #3756)

auth login browser flow prints a fatal stack trace when swamp-club cannot be reached

Scheduled flaky-test detection is failing

v3 first clone and restore of a bucket datastore take 2.6-3x v2, missing #3418's 1.5x target (found by #3756)

CVE pages: horizontally scrolling code blocks are not keyboard-reachable

Homepage club plate link has no visible keyboard focus

Scheduled flaky-test detection is failing

swamp update downloads before failing in a sticky directory with another user's binary, and blames an immutable flag

Docs: extra-headers how-to systemd snippet puts a path in SWAMP_SERVE_EXTRA_HEADERS

Docs: install-swamp says SWAMP_HOME moves everything, but agent skill folders stay in the user's home folder

Docs: dashboard Approvals view shows run context, reason and per-viewer decide state

v3 bucket outage: a workflow run whose checkpoint committed is told nothing was committed, three outage errors have no stable code, and exit takes 15-60 s (found by #3756)

v3 identity on buckets: S3 checkouts with different endpoint spellings fork the datastore silently, and a hostname change blocks every write for ~5 minutes (found by #3756)

typed_workflow_passes_test child-sentinel case flakes under full-suite load: f.received.at(-1) reads the compare step's empty payload

extension push drops local modules when the repo under a node_modules folder is reached through a symlinked path (macOS /var, the swamp-club#3782 fix)

Teach defineBehavior in the extension skill now that swamp-testing 0.20261013.52 exports it

Docs: issue redaction keeps git ids and non-address colon runs, and asks before posting (swamp-club#3686)

Docs: worker daemon enable reference is missing --ca-cert, --concurrency and SWAMP_SERVE_EXTRA_HEADERS

v3 on S3/GCS: a write whose commit cannot reach the bucket prints "succeeded" on --json, exits 1, and is lost (confirm-before-output, found by #3756)

swamp-club /cve/dirtyfrag page shows --input targetHost/sshUser/sshKey/dockerContainer, which the CLI rejects

v3: datastore config migrate skips every workflow that has run (name-committed), and they are then "not found" (found by #3756)

@swamp/cve/dirtyfrag report suggests commands for dirtyfrag-scanner whatever the model is named

v3 on a bucket: after a commit fails because the bucket is unreachable, the closing sync still waits its full 60 s before the command exits

Interactive history and output search pickers cannot open a run under a non-default datastore

v3 recovery sweep: dot-prefixed data names escape it, HFS+ non-ASCII names are deleted by it, and an old mtime keeps a dropped write's file (bug hunt group 8)

Source glob expansion crashes when a directory under the glob parent disappears mid-walk

v3 chunk barrier on macOS: use F_BARRIERFSYNC (ordering) instead of F_FULLFSYNC (full drive-cache flush), cutting ~10-25 ms (200 ms+ on a busy disk) per command

extension_source_add_test globs the real home directory and flakes when a file in ~ disappears mid-walk

v3 join check refuses a fresh managedConfig checkout on S3/GCS: the datastore extension's auto-install writes the managed lockfile first (regression of #3690)

Docs: doctor datastores reference for the v3 content check and lost format marker repair

Docs: step dataOutputOverrides vaultName and the field-by-field override merge (swamp-club#3606)

Docs: data name portability rule and warning

Docs: vault create and vault migrate refuse a local_encryption name with leftover storage

v3 namespaces: cross-namespace reads return the reader's own content, a namespaced local tier is overwritten by solo commits, and namespaces can collide with swamp's own .swamp/ directories (bug hunt group 7)

v3: a checkout holding v2 records joins a v3 datastore through .swamp.yaml unguarded, overwriting its records and diverging for good (bug hunt group 5)

v3 on S3/GCS: the namespace refusal names setup filesystem, and an unreadable marker on a format-3 repo says "will not open … as format 2" (found by swamp-uat#715)

v3 on S3/GCS: setup on a prefix holding only swamp's format-3 marker fails with "this swamp release can't open" format 3 (found by swamp-uat#715)

Workflow runs ignore defaultVault: step sensitive outputs land in the first vault, and validate predicts a vault the run doesn't use

v3 local durability: chunks aren't fsynced before the commit naming them, so a power cut can wedge every checkout; torn writer and format markers don't heal (bug hunt group 4)