#2864 CI: run swamp-uat's datastore suite against the PR binary for PRs touching datastore code

Background (read this first)

swamp is about to rework its datastore layer. The design proposal is design/enablers/datastore-commit-log.md on the datastore-rework branch. It is not merged; §2 of it is a write-path inventory.

How the datastore works today.

• Repositories write files into the repo's .swamp/ directory, or into the datastore cache dir for a custom datastore. They then call a mark hook, markDirty(relPath?), which reaches DatastoreSyncService (src/domain/datastore/datastore_sync_service.ts, interface at :168, the eight-rule markDirty contract at :199-257).
• A caller then flushes, through one of four paths:
  • acquireModelLocks().flush (src/cli/repo_context.ts:1723);
  • the global coordinator flushDatastoreSync (src/cli/mod.ts:2251);
  • pushManagedConfigChanges (src/cli/managed_config_sync.ts:45/153);
  • serve's pushChangedToRemote (src/serve/handlers/shared.ts:83-97) under the sync gate.
• The S3 and GCS sync implementations live in swamp-extensions, not here. They own the dirty set, the .datastore-sync-state.json sidecar, _index shards and _meta.json.
• A filesystem datastore has no sync service at all (repo_context.ts:1170-1178).
• Each repo has a SQLite catalog, .swamp/data/_catalog.db, that DataQueryService reads.
• Serve runs pollers that pull peers' changes: Config, AccessData, RuntimeData.

What changes. The refactor lands on main in phases:

• Phase 1: a UnitOfWork port with a legacy adapter, so repositories stage writes instead of calling the hook directly.
• Phase 2: libswamp use cases own the unit of work, and CLI commands and serve stop calling markDirty/pushChanged themselves.
• Later phases: a new commit-log engine behind an opt-in format.

Phases 1 and 2 must not change any behaviour.

Why this issue exists. Before any of that lands, this repo's own tests have to pin today's behaviour, so a dropped mark, a lost delete or a missed catalog refresh fails a test instead of reaching users. An audit in September 2026 found:

• no in-memory remote to test sync against;
• no test that every file a repository changes gets marked, and several real unmarked paths;
• no two-repo propagation test;
• no ratchet on the write seams Phase 1 will move;
• the sync-service conformance suite only checks method shapes.

This issue is one of a set. The tracking issue is listed under Related. End-to-end CLI tests for the same risks are in swamp-club/swamp-uat #481-#485, #487, #490 and #492.

Rules (from AGENTS.md)

• Unit tests are in-process only: no subprocesses, no process-global mutation; use withMockedEnv.
• Integration tests in integration/ wire real components on a temp filesystem and must not spawn the CLI.
• Registry-registered test types use a per-run name built with crypto.randomUUID(), or invalidateType in a finally.
• No fixed sleeps (use waitFor from @swamp-club/swamp-testing), no wall-clock assertions, and Deno.utime for mtimes.
• Use @std/path for paths and assertPathEquals for path comparisons, since tests run on Windows too.
• New files need the AGPLv3 header (deno run license-headers).
• Pin today's behaviour, including known gaps. When today's behaviour is a bug, assert it as it is, with a comment naming the bug. Prefer an explicit pinned list checked with assertPinnedSet, so a later fix forces the test to be updated on purpose. Do not fix production code in these issues unless the issue says so.

Goal

Make swamp PRs that touch datastore code run swamp-uat's datastore suite against the PR's own binary, on the S3 (ministack) and GCS (fake-gcs) emulators, before merge. Today that suite runs only after a release. Every datastore regression from the refactor would otherwise surface after merge at best.

Current state

• No tests run in PR CI. Verification runs locally as swamp workflows in verification/workflow-verify-{build,reviews,skills}.yaml, and deno run build-attestation (scripts/build_attestation.ts) produces an attestation.
• .github/workflows/ci.yml:
  • validate-attestation (:487-736) checks the attestation;
  • auto-merge (:738-766) needs claude-adversarial-review, claude-ci-security-review, claude-review-integrity and validate-attestation.
• Branch protection has no required status checks.
• release.yml:176-182 sends repository_dispatch run-uat with {version} to swamp-club/swamp-uat, after merge only.
• In swamp-uat (private repo):
  • .github/workflows/uat-datastore.yml installs a released binary and runs deno task uat:datastore (= tests/cli/datastore/) on ministack and fake-gcs from docker/datastore-compose.yml;
  • tests use ignore: !backend.isAvailable(), so a missing env passes silently;
  • the harness accepts SWAMP_BINARY_PATH (src/cli/helpers/swamp_runner.ts).
• Do not reuse run-uat: swamp-uat's uat.yml:135-160 promotes the binary to stable on that event.

Work (recommended option: a job in swamp's ci.yml)

1. Add a datastore output to the changes job in .github/workflows/ci.yml (:14-56), covering:

   • src/domain/datastore/**, src/libswamp/datastores/**, src/infrastructure/persistence/**;
   • src/cli/commands/datastore*, src/cli/repo_context.ts, src/cli/managed_config_sync.ts;
   • src/serve/sync_gate.ts, src/serve/*_poller.ts;
   • packages/testing/datastore*, deno.json, deno.lock.

2. Add a uat-datastore job, run if: needs.changes.outputs.datastore == 'true', only for same-repo, non-draft PRs. It:

   • checks out the PR head;
   • sets up Deno from .tool-versions (deno-version-file);
   • runs deno task compile;
   • checks out swamp-club/swamp-uat at a pinned ref with a read token;
   • runs docker compose -f docker/datastore-compose.yml up -d --wait ministack fake-gcs;
   • runs deno task uat:datastore twice, with the S3 env and then the GCS env (copy the env blocks from swamp-uat's uat-datastore.yml), with SWAMP_BINARY_PATH pointing at the compiled binary;
   • fails if zero tests ran, or if any test was ignored for a missing backend.

   Once swamp-uat #483 lands, also run the non-datastore/ files that contain [backend] tests, using the file list that issue produces.

3. Add the job to auto-merge.needs (:740-746). A skipped job does not trip !failure().

4. Update the fitness tests:

   • integration/toolchain_pins_rules_test.ts:44-51 (EXPECTED_SETUP_STEPS, add "ci.yml": 1);
   • check integration/verification_harness_rules_test.ts:288, whose regex reads the trust_root: block; keep its terminating blank line.

5. Pin third-party actions by SHA (scripts/audit_actions.ts:275).

6. Secrets: a token that can read the private swamp-uat repo. Check whether UAT_TRIGGER_TOKEN has that scope; otherwise add one. Possibly SWAMP_API_KEY too, for pulling @swamp/s3-datastore / @swamp/gcs-datastore from the registry (swamp has SWAMP_CLUB_API_KEY). The emulators need no cloud credentials.

7. Docs: add a sentence to agent-constraints/verification-conventions.md, which says CI validates attestations and does not run tests (:516-521), explaining this exception.

This change touches .github/workflows/**, so claude-ci-security-review and actionlint will run on it.

Alternatives considered:

• A guarded step in verification/workflow-verify-build.yaml. This is a trust-root change, needs Docker and a swamp-uat checkout on every developer machine, and is only as trustworthy as the attestation.
• A new dispatch event to swamp-uat that posts a commit status back. This needs changes in both repos and a statuses:write token.

Done when

A swamp PR touching src/infrastructure/persistence/** runs the datastore suite on both emulators against its own binary, auto-merge waits for it, and a PR not touching those paths skips it.

Dependencies

Blocked by: swamp-uat#483
Blocks: nothing
Full dependency graph and waves: swamp-club#2865.

Related

• Tracking issue: swamp-club#2865 (https://swamp-club.com/lab/2865).
• Calls the uat:datastore:matrix task and the require-backends mode added in swamp-uat#483.