Skip to main content
← Back to list
01Issue
FeatureShippedExtensionsPublic
Assigneesstack72

Relationships

#3508 Datastore rework Phase 4 G: S3 and GCS extensions implement replicaTarget() for v3 Litestream replication

Opened by stack72 · 10/15/2026· Shipped 10/16/2026

Background

swamp is reworking its datastore layer (tracking: swamp-club#2865). Phase 4E puts the v3 format on S3 and GCS:

  • a checkout's commits.db is replicated by Litestream to the datastore's bucket, under <prefix>/_litestream/<fence>/;
  • chunks go through storage/1;
  • the writer lease and lineage records live in the datastore-wide control plane.

@swamp/s3-datastore and @swamp/gcs-datastore 2026.10.15.2 (swamp-club#3395, #3492) already provide the control plane and storage/1.

The last piece the extensions owe is telling swamp where Litestream replicates: the core contract replicaTarget(), defined in swamp-club#3507. swamp-extensions has issues turned off, so this is tracked here.

What is needed

  1. replicaTarget() on the S3 provider: { type: "s3", bucket, prefix, region?, endpoint?, forcePathStyle? } from the datastore's config. prefix is the same datastore root that storage() and datastoreControlPlaneStore() use.
    • Covers AWS and every S3-compatible endpoint the extension supports (R2, MinIO, DigitalOcean Spaces and others).
  2. replicaTarget() on the GCS provider: { type: "gcs", bucket, prefix }.
    • Refuse when apiEndpoint is set, with a clear error: Litestream has no custom GCS endpoint, so it would replicate to real GCS instead of the emulator.
  3. No credentials in the descriptor. Both providers resolve credentials from the default chains (AWS SDK chain, Google ADC), and Litestream does the same. Document that in the README: whatever credentials the datastore uses must also be visible to the Litestream sidecar, through the same environment or shared config.
  4. Run assertReplicaTargetConformance from @swamp-club/swamp-testing once core publishes it.
  5. Bump both manifests. The README and manifest should say that Phase 4E names this version as the minimum.

Tests

  • Unit tests: the descriptor for each config shape (region, endpoint, path style, empty and nested prefixes); GCS's refusal with apiEndpoint.
  • The conformance suite.
  • An integration check against ministack and fake-gcs-server where SWAMP_TEST_*_ENDPOINT is set: a Litestream replica configured from the descriptor lands under <prefix>/_litestream/. GCS only without a custom endpoint, so this may be S3-only locally. Say so.

Blocked by: swamp-club#3507.

02Bog Flow
✓OPEN✓TRIAGED✓IN PROGRESS✓SHIPPED+ 2 MOREISSUE_LINKED+ 7 MOREREVIEW+ 29 MOREPR_LINKED+ 2 MORESESSION_SUMMARIZED

Shipped

10/16/2026, 3:22:55 PM

Click a lifecycle step above to view its details.

03Sludge Pulse
stack72 assigned stack7210/15/2026, 10:45:15 PM
stack72 linked sibling of #337710/15/2026, 10:46:42 PM
stack72 removed sibling of #337710/16/2026, 2:55:39 PM
Editable. Press Enter to edit.

system commented 10/15/2026, 6:50:52 PM

Classified automatically when this issue was filed.

  • Source: Extensions

If you feel this classification is incorrect, add a ripple to tell us so.

stack72 commented 10/15/2026, 10:42:14 PM

Scope addition (swamp-club#3531): please also implement replicaPaths() on both providers in the same release as replicaTarget(). It's an optional capability scoped to <prefix>/_litestream/, with list(signal) (fence directories present) and removePath(fence, signal) (delete every object under <prefix>/_litestream/<fence>/, paging, idempotent, never outside that path; refuse a fence that isn't the 12-digit padded form). Core needs it to collect abandoned lineage paths on S3 and GCS. Without it, chunk GC stays deferred forever on buckets (swamp-club#3491's rule). The core contract and conformance land in #3531; replicaTarget() can start now, since @swamp-club/swamp-testing 0.20261015.56 already ships assertReplicaTargetConformance. Pick up replicaPaths() once #3531 publishes its conformance suite.

Sign in to post a ripple.