#3507 Datastore rework Phase 4: a provider-supplied replica target (replicaTarget()) so v3 replicates to the datastore's own bucket, plus conformance

Background (read this first)

swamp is reworking its datastore layer (tracking: swamp-club#2865). In the new format, v3, every change is a commit in a single-writer SQLite commits.db. Content is stored as content-addressed chunks, and each checkout rebuilds its files from the commits. v3 is opt-in today for filesystem datastores (swamp-club#3279), and swamp-uat's filesystem-v3 backend tests it in CI.

Phase 4 is "solo mode on buckets":

• the CLI is the single writer of a local commits.db;
• Litestream replicates it to the bucket;
• chunks go to the bucket through storage/1;
• a fenced writer lease makes sure only one machine writes.

Shipped so far:

• A, swamp-club#3357: replication cost, choosing an idle sidecar.
• B, swamp-club#3358: the pinned, hash-checked Litestream binary.
• C, swamp-club#3359: the fenced writer lease (src/domain/datastore_v3/writer_lease.ts, src/infrastructure/datastore_v3/v3_writer_lock.ts). Every CLI write command takes a new fence, serve keeps one for its lifetime, and commits.db keeps a store-wide writer_fence that refuses lower fences.
• D1, swamp-club#3416: a dormant Litestream sidecar (src/infrastructure/datastore_v3/litestream/, src/domain/datastore_v3/litestream_replication.ts and replication_lineage.ts).
  • A write command confirms its commits are replicated (POST /sync with wait, replicated_txid >= txid, then the lease check) before it succeeds.
  • Replication follows a lineage: the replica path is _litestream/<fence>/, named by the fence at which this copy's replication began. Another writer's fence restarts it on a new path. The lineage is recorded in lineage.json beside commits.db, with abandoned ones in previousLineages.
  • Nothing replicates unless the internal SWAMP_V3_REPLICA setting (replica_setting.ts, tests only) names a file or S3 replica.
• D2, swamp-club#3482: restore, refresh and takeover, also dormant.
  • While replicating, each checkout keeps its own commits.db under .swamp/_datastore_v3_copies/.
  • Lineages are published under _control/lineage/<fence>/ (started.json, head-<writer fence>.json, confirmed.json).
  • A restore truncates to the published head's high-water mark.
  • See "Phase 4D2" in datastore-litestream.md.

The decisions are in design/enablers/datastore-litestream.md ("Decision", "Restore, failover and fences (Q6)", "Phase 4D1") and in design/enablers/datastores.md (the "v3 …" sections).

• F, swamp-club#3490: the commit outbox. D3, swamp-club#3489: serve confirms each write. swamp-club#3491: D2's follow-ups.
• The extensions, @swamp/s3-datastore and @swamp/gcs-datastore 2026.10.15.2 (swamp-club#3395, #3492):
  • a writable datastore-wide control plane (datastoreControlPlaneStore(), datastoreWideLease) that keeps _control/writer/, _control/lineage/ and _control/datastore-format at the root;
  • storage/1 chunk storage.

Goal

Let the datastore itself tell swamp where Litestream replicates, so Phase 4E can replace the test-only SWAMP_V3_REPLICA setting with the datastore's own bucket.

Today the replica comes from SWAMP_V3_REPLICA (src/infrastructure/datastore_v3/litestream/replica_setting.ts: {type: "file", path} or {type: "s3", bucket, path, endpoint?, region?}). Only the extension knows how its config maps to a bucket location, so core must not parse a provider's config shape.

Design

1. A new optional provider method in the datastore provider contract (packages/testing/datastore_types.ts, and core's mirror of it): replicaTarget(): V3ReplicaTarget. It describes where this datastore's Litestream replica lives:

   • S3 or S3-compatible: { type: "s3", bucket, prefix, region?, endpoint?, forcePathStyle? };
   • GCS: { type: "gcs", bucket, prefix }.

   prefix is the datastore root's key prefix, ending without a slash or empty. Core appends _litestream/<fence>/ (replicaPathFor), so the provider never builds replica paths itself.

2. No secrets in the descriptor. Both first-party extensions resolve credentials from the default chains: the AWS SDK chain (env, shared config and SSO, container or instance identity) and Google ADC. Litestream resolves them the same way. Core decides which environment variables Litestream's supervisor inherits. Record that list in datastore-litestream.md (D1 noted 4E narrows the environment):

   • S3: AWS_*, HOME for ~/.aws, PATH;
   • GCS: GOOGLE_APPLICATION_CREDENTIALS, CLOUDSDK_*, HOME, PATH.

   A provider whose credentials don't come from those chains can't be replicated. Say so in the method's doc.

3. Refusals: a provider without replicaTarget(), or one that returns a target Litestream can't reach, can't host a v3 datastore. E refuses --format 3 setup for it with a clear message.

   • Example: GCS with a custom apiEndpoint. Litestream has no equivalent, so the GCS extension refuses in replicaTarget() rather than returning a target that silently replicates to real GCS.

4. Wire it dormant. Add a resolver that turns a provider's V3ReplicaTarget into D1's V3ReplicaSetting and Litestream config, used only behind the existing dormant path. SWAMP_V3_REPLICA still wins in tests. Phase 4E switches the source.

5. Conformance: assertReplicaTargetConformance in packages/testing. The target parses, prefix matches the datastore root that storage() and datastoreControlPlaneStore() write under, and it carries no credential fields.

6. Also extend assertWritableDatastoreControlPlaneStoreConformance so its namespaced-push case checks a lineage/ record stays at the root next to writer/. The extensions already do this in their own tests (swamp-extensions #497); the suite should hold every provider to it.

Tests

• Unit tests for the descriptor types, the resolver (S3, S3-compatible with endpoint and path style, GCS) and the refusal of a missing method.
• The conformance suites, run against the in-memory remote.
• Dormancy: nothing calls replicaTarget() outside tests and the dormant path; a fitness rule pins it.

Blocked by: nothing. The extensions' implementation is a separate Lab issue. Phase 4E waits for both.