#3915 Datastore rework Phase 5 (team mode) 1/3: serve's datastore API — commit, chunk and catch-up endpoints, authorisation on each op's intent, principal stamping and actionable conflicts

Background

swamp's datastore rework (tracking: swamp-club#2865) finished solo mode (Phase 4) on 2026-10-25: v3 datastores on filesystem, S3 and GCS with a commit log (commits.db), content-addressed chunks, a fenced writer lease and Litestream replication, with one writer at a time. Phase 5 is team mode. A team writes through swamp serve, which holds the writer lease, and laptops and CI hold a serve token instead of storage credentials. v3 on buckets isn't announced until team mode works.

Team mode's client work (p5-client in the plan) is three issues:

1. this one, serve's datastore API;
2. the CLI's serve upstream (serverAddress);
3. swamp datastore claim and release.

This issue is the serve side only. It's the security boundary for everything else, so it comes first, with no CLI change.

What exists today (inventory of main, 2026-10-25):

• serve takes the fenced v3 writer lease and commits its own requests through the embedded datastore service (holdV3Writer, confirmRequest, holdRepliesUntilCommitted);
• commits carry typed intents (src/domain/datastore_v3/intent.ts) and a principal field documented as "set by serve only";
• intent.ts says "Serve authorises on the intent, never on the path", but nothing does yet; serve checks access per request (authorizeOrReject, src/serve/handlers/shared.ts);
• serve has no commit or chunk endpoints: the WebSocket protocol has no commits.*/chunks.* messages, and src/serve/data_plane.ts is only for remote-execution workers.

What to build

1. The datastore API

Under serve's versioned API, per namespace (/api/v1/datastore/{namespace}/), bearer-authenticated with serve's existing token and OAuth modes:

Follow data_plane.ts's pattern: stream content in and out, and never buffer a whole chunk in memory. Hash each chunk while storing it, and reject a mismatch.

2. Per commit, in order

1. Authenticate the caller, as serve does today.
2. Authorise every op on its intent, never its path. Add a pure domain service that maps an intent to a resource and an action (data.save on model aws-ec2/web → model:… write), checked with the existing decision service. The binding rules raised in the reviews of swamp-club#3233 and #3236:
   • an op's record key must fall under its intent's aggregate paths;
   • a table of which intents may ride a put and which a tombstone;
   • data-version ops must be tied to a data.save or rename intent in the same commit;
   • lease ops come only from the lease machinery;
   • grants, groups, tokens and server-token definitions need admin actions, and a grant record written as plain data is refused.
3. Rate-limit per principal with serve's existing limiter.
4. Store the chunks, streaming.
5. Stamp principal and writer from the authenticated caller, overwriting anything the client sent.
6. Commit through serve's own datastore service: same preconditions, fence and outbox as today.
7. Reply with {seq}.

3. Conflicts a user can act on

• A stale expectedSeq: 409 with the current seq, so the client can catch up and retry.
• A failed precondition (for example expect.sha256 no longer matches): 409 {key, changedBy, changedAt, seq}, worded like a rejected git push.
• Fenced, lease lost, or storage unreachable: the existing codes (WRITER_FENCED_CODE, datastore_v3_commit_unreachable and the rest), as replicationErrorForClient maps them today.

4. The commit store under serve (folded in)

From the review of swamp-club#3236:

• commit-store appends block the event loop for their fsync;
• a long-cached busy partition collects every txnId since its snapshot.

Bound both before clients write through serve. Measure serve's per-request cost before and after, in-process with an A/A control.

5. Design

Add a short "Team mode: serve's datastore API" section to design/enablers/datastores.md: the endpoints, the intent → resource/action table with the binding rules, stamping, and the conflict replies. The commit-log proposal this follows isn't in the repo, so this section becomes the in-repo design.

Out of scope (later issues)

• the CLI using this API (serverAddress upstream);
• swamp datastore claim and release;
• commits.subscribe, follower nodes and failover;
• model leases;
• offline queue and rebase.

Done when

• An in-process client against a port-0 serve can, through the API:
  • commit, with chunks;
  • check which chunks serve holds;
  • catch up;
  • read a chunk with Range;
  • list changes since a cursor, filtered to what it may read.
• The authoriser is tested as a domain service:
  • unit tests for every intent kind;
  • a property test that no key outside an intent's aggregate paths is ever authorised;
  • a test that a grant record written as plain data is refused.
• Each binding rule has a test that fails without it.
• Stamping: a client-supplied principal or writer is overwritten.
• Conflicts: each 409 shape and each mapped error code has an integration test.
• Streaming and hashing: a chunk whose hash doesn't match is rejected, and nothing is committed.
• The commit-store bounds are measured before and after.
• No behaviour change for solo mode or v2: the swamp-uat suites pass unchanged.

Blocked by: nothing. Phase 4 is closed.