Postgres Datastore
Stores swamp runtime data in PostgreSQL with row-based distributed locking. Compatible with AWS RDS, Aurora, and Aurora Serverless v2. Uses fencing tokens for split-brain safety across failover events. Emits OpenTelemetry spans for SQL operations, lock acquisition/release, and push/pull sync so datastore activity is visible in traces when a TracerProvider is configured.
2026.07.30.1
Fixed: Subtree lookup query in the incremental push path forced sequential
scans on the files table. The WHERE path = $1 OR path LIKE $2 pattern made
PostgreSQL unable to use the btree primary key for the LIKE branch, causing
O(n^2) write latency as the catalog grew past ~30K files.
Changed: The subtree query now uses UNION ALL to split the exact-match and
prefix-match predicates into independent branches, each with its own query plan.
A new text_pattern_ops btree index on the path column is created
automatically on first connect, enabling index scans for all prefix-anchored
LIKE queries (both the push and pull paths).
Upgrade note: The new index is created via CREATE INDEX IF NOT EXISTS on
startup. For large tables (>100K rows) the initial index build may take a few
seconds on first connection after upgrade. Subsequent connections are
instantaneous. No manual migration required.
Stores swamp runtime data in PostgreSQL with row-based distributed locking. Compatible with AWS RDS, Aurora, and Aurora Serverless v2.
Config Fields
| Field | Type | Description |
|---|---|---|
| connectionString | string | PostgreSQL connection URI (supports RDS, Aurora, Aurora Serverless v2) |
| schema | string | PostgreSQL schema for swamp tables |
| ssl | enum | SSL mode: disable (no TLS), require (TLS without CA verify), verify-ca (TLS with CA bundle) |
| sslCaPath? | string | Path to CA certificate bundle (e.g., RDS global-bundle.pem). Required when ssl=verify-ca. |
2026.07.30.1
Fixed: Subtree lookup query in the incremental push path forced sequential
scans on the files table. The WHERE path = $1 OR path LIKE $2 pattern made
PostgreSQL unable to use the btree primary key for the LIKE branch, causing
O(n^2) write latency as the catalog grew past ~30K files.
Changed: The subtree query now uses UNION ALL to split the exact-match and
prefix-match predicates into independent branches, each with its own query plan.
A new text_pattern_ops btree index on the path column is created
automatically on first connect, enabling index scans for all prefix-anchored
LIKE queries (both the push and pull paths).
Upgrade note: The new index is created via CREATE INDEX IF NOT EXISTS on
startup. For large tables (>100K rows) the initial index build may take a few
seconds on first connection after upgrade. Subsequent connections are
instantaneous. No manual migration required.
2026.07.29.1
Added: Monotonic commitSeq via PostgreSQL sequence. Pull fast-path now
compares a single integer instead of parsing timestamps, eliminating clock-skew
vulnerabilities across concurrent writers.
Changed: Dirty-path pushes are batched into a single transaction (was N
separate BEGIN/COMMIT pairs). One watermark write instead of N. Expected
10-50x latency improvement for incremental pushes.
Changed: Pull captures commitSeq before the metadata scan (not after) to
prevent TOCTOU races where concurrent pushes during the fetch window could be
silently missed.
Changed: Content-fetch batches during pull run up to 3 in parallel via
Promise.all.
Changed: Manifest query for push uses batch path lookups (IN clause) instead
of a full table scan when lastPulledAt provides a bound.
Added: Tombstone garbage collection — every push transaction deletes tombstones older than 7 days, preventing unbounded table growth.
2026.07.27.1
Changed: Bump @opentelemetry/api 1.9.0 → 1.9.1
2026.07.25.1
Added: OpenTelemetry spans for every layer of the datastore. Each SQL round
trip emits a span named for the operation it performs
(PostgreSQL acquireLock, PostgreSQL scanFileMetadata,
PostgreSQL commitPushTransaction, …) carrying db.system.name,
db.operation.name, db.collection.name, and returned row counts. The lock
emits postgres-datastore lock acquire / release / withLock / inspect /
forceRelease, with acquire recording wait duration and whether it contended.
The sync service emits postgres-datastore pullChanged / pushChanged /
hydrateFile / preparePush / commitPush with file counts and fast-path
indicators.
Added: Retries are recorded as retry span events on the enclosing
operation — both the transient-error backoff in retryable and the lock
contention loop.
Changed: pushChanged now reports datastore.files_pushed and
datastore.files_deleted separately, and pullChanged likewise separates
downloads from local deletions. The internal counters increment for both writes
and tombstones, so reporting either as a file count would have overstated it.
The values returned to callers are unchanged.
Changed: Nothing else observable without tracing configured. The extension
depends on @opentelemetry/api only; the host process owns the
TracerProvider, and every span is a no-op when none is registered.
Note: The existing SWAMP_PG_SYNC_TRACE=1 phase tracer is untouched and
still writes its own timing lines to stderr. The two are independent — enable
either, both, or neither.
Note on secrets: Statement text and bound parameters are never recorded. Query parameters carry file content and the connection string carries a password, so span attributes hold only hand-written operation labels, table names, and counts.
2026.07.27.1
Changed: Bump @opentelemetry/api 1.9.0 → 1.9.1
2026.07.25.1
Added: OpenTelemetry spans for every layer of the datastore. Each SQL round
trip emits a span named for the operation it performs
(PostgreSQL acquireLock, PostgreSQL scanFileMetadata,
PostgreSQL commitPushTransaction, …) carrying db.system.name,
db.operation.name, db.collection.name, and returned row counts. The lock
emits postgres-datastore lock acquire / release / withLock / inspect /
forceRelease, with acquire recording wait duration and whether it contended.
The sync service emits postgres-datastore pullChanged / pushChanged /
hydrateFile / preparePush / commitPush with file counts and fast-path
indicators.
Added: Retries are recorded as retry span events on the enclosing
operation — both the transient-error backoff in retryable and the lock
contention loop.
Changed: pushChanged now reports datastore.files_pushed and
datastore.files_deleted separately, and pullChanged likewise separates
downloads from local deletions. The internal counters increment for both writes
and tombstones, so reporting either as a file count would have overstated it.
The values returned to callers are unchanged.
Changed: Nothing else observable without tracing configured. The extension
depends on @opentelemetry/api only; the host process owns the
TracerProvider, and every span is a no-op when none is registered.
Note: The existing SWAMP_PG_SYNC_TRACE=1 phase tracer is untouched and
still writes its own timing lines to stderr. The two are independent — enable
either, both, or neither.
Note on secrets: Statement text and bound parameters are never recorded. Query parameters carry file content and the connection string carries a password, so span attributes hold only hand-written operation labels, table names, and counts.
2026.07.25.1
Added: OpenTelemetry spans for every layer of the datastore. Each SQL round
trip emits a span named for the operation it performs
(PostgreSQL acquireLock, PostgreSQL scanFileMetadata,
PostgreSQL commitPushTransaction, …) carrying db.system.name,
db.operation.name, db.collection.name, and returned row counts. The lock
emits postgres-datastore lock acquire / release / withLock / inspect /
forceRelease, with acquire recording wait duration and whether it contended.
The sync service emits postgres-datastore pullChanged / pushChanged /
hydrateFile / preparePush / commitPush with file counts and fast-path
indicators.
Added: Retries are recorded as retry span events on the enclosing
operation — both the transient-error backoff in retryable and the lock
contention loop.
Changed: pushChanged now reports datastore.files_pushed and
datastore.files_deleted separately, and pullChanged likewise separates
downloads from local deletions. The internal counters increment for both writes
and tombstones, so reporting either as a file count would have overstated it.
The values returned to callers are unchanged.
Changed: Nothing else observable without tracing configured. The extension
depends on @opentelemetry/api only; the host process owns the
TracerProvider, and every span is a no-op when none is registered.
Note: The existing SWAMP_PG_SYNC_TRACE=1 phase tracer is untouched and
still writes its own timing lines to stderr. The two are independent — enable
either, both, or neither.
Note on secrets: Statement text and bound parameters are never recorded. Query parameters carry file content and the connection string carries a password, so span attributes hold only hand-written operation labels, table names, and counts.
updated labels
2026.07.18.1
Changed: Version bump only, no code changes.
2026.07.16.1
Changed: Internal-only version bump. PR #183 added a fmt:check task to deno.json, which is not part of the published bundle — this release's published content is identical to 2026.06.30.1.
2026.07.16.1
Changed: Internal-only version bump. PR #183 added a fmt:check task to deno.json, which is not part of the published bundle — this release's published content is identical to 2026.06.30.1.
2026.06.30.1
Two-Phase Datastore Sync
Adds preparePush/commitPush two-phase sync protocol alongside the existing
fullWalkPush and pushOneRel single-shot paths.
Added
TwoPhaseSyncServiceinterface withpreparePush()andcommitPush()methodsPushManifestopaque branded type for safe manifest passing between phasespreparePush()captures sidecar snapshot and collects diff without transactioncommitPush()executes batched inserts in a transaction, clears sidecarcapabilities()now reportstwoPhaseSync: true
Changed
- Extracted
collectFullWalkDiff()andcollectOneRelDiff()helper functions - Refactored
fullWalkPush/pushOneRelto use extracted helpers (behavior preserved) - No breaking changes to existing sync behavior
- Has README or module doc2/2earned
- README has a code example1/1earned
- README is substantive1/1earned
- Most symbols documented1/1earned
- No slow types (deprecated)1/1earned
- Dependencies pass trust audit2/2earned
- Has description1/1earned
- Platform support declared (or universal)2/2earned
- License declared1/1earned
- Verified public repository2/2earned