Skip to main content

Postgres Datastore

@webframp/postgres-datastorev2026.08.28.1· 13d agoDATASTORES
01README

Stores swamp runtime data in PostgreSQL with row-based distributed locking. Compatible with AWS RDS, Aurora, and Aurora Serverless v2. Uses fencing tokens for split-brain safety across failover events. Emits OpenTelemetry spans for SQL operations, lock acquisition/release, and push/pull sync so datastore activity is visible in traces when a TracerProvider is configured.

02Release Notes

2026.08.28.1

Changed: Normalized the extension license to Apache-2.0 and corrected the copyright holder to "Sean Escriva". Extensions that previously shipped an MIT LICENSE.md are now Apache-2.0, consistent with the repository root and every other extension. No code or behavioral changes.

Upgrade note: License text only. No API, schema, or runtime behavior changed.

2026.08.26.3

Fixed: Restored inline npm:zod@4.4.3 import specifiers so the registry quality scorer can resolve dependencies and score the extension. An earlier release used a bare "zod" import-map specifier, which published but scored as unscored.

Changed: Retained explicit compilerOptions.strict in deno.json. No behavioral or schema changes.

03Datastores1
PostgreSQL Datastoreconfigurable
@webframp/postgres-datastorepostgres_datastore/mod.ts

Stores swamp runtime data in PostgreSQL with row-based distributed locking. Compatible with AWS RDS, Aurora, and Aurora Serverless v2.

Config Fields

FieldTypeDescription
connectionStringstringPostgreSQL connection URI (supports RDS, Aurora, Aurora Serverless v2)
schemastringPostgreSQL schema for swamp tables
sslenumSSL mode: disable (no TLS), require (TLS without CA verify), verify-ca (TLS with CA bundle)
sslCaPath?stringPath to CA certificate bundle (e.g., RDS global-bundle.pem). Required when ssl=verify-ca.
04Previous Versions17
2026.08.26.3

2026.08.26.3

Fixed: Restored inline npm:zod@4.4.3 import specifiers so the registry quality scorer can resolve dependencies and score the extension. An earlier release used a bare "zod" import-map specifier, which published but scored as unscored.

Changed: Retained explicit compilerOptions.strict in deno.json. No behavioral or schema changes.

2026.08.26.1

2026.08.26.1

Changed: Normalized deno.json configuration for repo-wide consistency: added explicit compilerOptions.strict and migrated zod dependency to the import map (bare "zod" specifier instead of inline npm:zod@4.4.3). No behavioral changes — runtime resolution is identical.

2026.08.25.1

2026.08.25.1

Changed: Updated labels for improved extension discoverability. Added cross-cutting category labels (security, observability, finops, infrastructure, networking, compliance, devops, ai, incident-response) where applicable.

updated labels

2026.08.24.1

2026.08.24.1

Added Troubleshooting section documenting auto-created infrastructure, silent heartbeat failure, connection pool behavior, retryable PG error codes, sidecar bulk invalidation, read replica detection, and SSL CA requirements.

2026.08.21.1

2026.08.21.1

Changed: SQL failures now say which operation and table they were acting on instead of surfacing the bare driver error (e.g. "relation does not exist" now reads as PostgreSQL insertFile on "swamp.files" failed: relation does not exist). The original error's name and Postgres error code are preserved, so retry classification and instanceof checks are unaffected. The lock heartbeat — deliberately unspanned to avoid flooding traces — now wraps a failed renewal with the lock key it was trying to renew, rather than raising a bare connection error with no indication of which lock was affected.

2026.08.20.1

Upgrade note: Pinned zod to exact version 4.4.3 (was unpinned range npm:zod@4). No behavioral changes — dependency version alignment only.

2026.08.20.1

2026.08.20.1

Upgrade note: Pinned zod to exact version 4.4.3 (was unpinned range npm:zod@4). No behavioral changes — dependency version alignment only.

2026.07.30.1

2026.07.30.1

Fixed: Subtree lookup query in the incremental push path forced sequential scans on the files table. The WHERE path = $1 OR path LIKE $2 pattern made PostgreSQL unable to use the btree primary key for the LIKE branch, causing O(n^2) write latency as the catalog grew past ~30K files.

Changed: The subtree query now uses UNION ALL to split the exact-match and prefix-match predicates into independent branches, each with its own query plan. A new text_pattern_ops btree index on the path column is created automatically on first connect, enabling index scans for all prefix-anchored LIKE queries (both the push and pull paths).

Upgrade note: The new index is created via CREATE INDEX IF NOT EXISTS on startup. For large tables (>100K rows) the initial index build may take a few seconds on first connection after upgrade. Subsequent connections are instantaneous. No manual migration required.

2026.07.29.1

2026.07.29.1

Added: Monotonic commitSeq via PostgreSQL sequence. Pull fast-path now compares a single integer instead of parsing timestamps, eliminating clock-skew vulnerabilities across concurrent writers.

Changed: Dirty-path pushes are batched into a single transaction (was N separate BEGIN/COMMIT pairs). One watermark write instead of N. Expected 10-50x latency improvement for incremental pushes.

Changed: Pull captures commitSeq before the metadata scan (not after) to prevent TOCTOU races where concurrent pushes during the fetch window could be silently missed.

Changed: Content-fetch batches during pull run up to 3 in parallel via Promise.all.

Changed: Manifest query for push uses batch path lookups (IN clause) instead of a full table scan when lastPulledAt provides a bound.

Added: Tombstone garbage collection — every push transaction deletes tombstones older than 7 days, preventing unbounded table growth.

2026.07.27.1

Changed: Bump @opentelemetry/api 1.9.0 → 1.9.1

2026.07.25.1

Added: OpenTelemetry spans for every layer of the datastore. Each SQL round trip emits a span named for the operation it performs (PostgreSQL acquireLock, PostgreSQL scanFileMetadata, PostgreSQL commitPushTransaction, …) carrying db.system.name, db.operation.name, db.collection.name, and returned row counts. The lock emits postgres-datastore lock acquire / release / withLock / inspect / forceRelease, with acquire recording wait duration and whether it contended. The sync service emits postgres-datastore pullChanged / pushChanged / hydrateFile / preparePush / commitPush with file counts and fast-path indicators.

Added: Retries are recorded as retry span events on the enclosing operation — both the transient-error backoff in retryable and the lock contention loop.

Changed: pushChanged now reports datastore.files_pushed and datastore.files_deleted separately, and pullChanged likewise separates downloads from local deletions. The internal counters increment for both writes and tombstones, so reporting either as a file count would have overstated it. The values returned to callers are unchanged.

Changed: Nothing else observable without tracing configured. The extension depends on @opentelemetry/api only; the host process owns the TracerProvider, and every span is a no-op when none is registered.

Note: The existing SWAMP_PG_SYNC_TRACE=1 phase tracer is untouched and still writes its own timing lines to stderr. The two are independent — enable either, both, or neither.

Note on secrets: Statement text and bound parameters are never recorded. Query parameters carry file content and the connection string carries a password, so span attributes hold only hand-written operation labels, table names, and counts.

2026.07.27.1

2026.07.27.1

Changed: Bump @opentelemetry/api 1.9.0 → 1.9.1

2026.07.25.1

Added: OpenTelemetry spans for every layer of the datastore. Each SQL round trip emits a span named for the operation it performs (PostgreSQL acquireLock, PostgreSQL scanFileMetadata, PostgreSQL commitPushTransaction, …) carrying db.system.name, db.operation.name, db.collection.name, and returned row counts. The lock emits postgres-datastore lock acquire / release / withLock / inspect / forceRelease, with acquire recording wait duration and whether it contended. The sync service emits postgres-datastore pullChanged / pushChanged / hydrateFile / preparePush / commitPush with file counts and fast-path indicators.

Added: Retries are recorded as retry span events on the enclosing operation — both the transient-error backoff in retryable and the lock contention loop.

Changed: pushChanged now reports datastore.files_pushed and datastore.files_deleted separately, and pullChanged likewise separates downloads from local deletions. The internal counters increment for both writes and tombstones, so reporting either as a file count would have overstated it. The values returned to callers are unchanged.

Changed: Nothing else observable without tracing configured. The extension depends on @opentelemetry/api only; the host process owns the TracerProvider, and every span is a no-op when none is registered.

Note: The existing SWAMP_PG_SYNC_TRACE=1 phase tracer is untouched and still writes its own timing lines to stderr. The two are independent — enable either, both, or neither.

Note on secrets: Statement text and bound parameters are never recorded. Query parameters carry file content and the connection string carries a password, so span attributes hold only hand-written operation labels, table names, and counts.

2026.07.25.1

2026.07.25.1

Added: OpenTelemetry spans for every layer of the datastore. Each SQL round trip emits a span named for the operation it performs (PostgreSQL acquireLock, PostgreSQL scanFileMetadata, PostgreSQL commitPushTransaction, …) carrying db.system.name, db.operation.name, db.collection.name, and returned row counts. The lock emits postgres-datastore lock acquire / release / withLock / inspect / forceRelease, with acquire recording wait duration and whether it contended. The sync service emits postgres-datastore pullChanged / pushChanged / hydrateFile / preparePush / commitPush with file counts and fast-path indicators.

Added: Retries are recorded as retry span events on the enclosing operation — both the transient-error backoff in retryable and the lock contention loop.

Changed: pushChanged now reports datastore.files_pushed and datastore.files_deleted separately, and pullChanged likewise separates downloads from local deletions. The internal counters increment for both writes and tombstones, so reporting either as a file count would have overstated it. The values returned to callers are unchanged.

Changed: Nothing else observable without tracing configured. The extension depends on @opentelemetry/api only; the host process owns the TracerProvider, and every span is a no-op when none is registered.

Note: The existing SWAMP_PG_SYNC_TRACE=1 phase tracer is untouched and still writes its own timing lines to stderr. The two are independent — enable either, both, or neither.

Note on secrets: Statement text and bound parameters are never recorded. Query parameters carry file content and the connection string carries a password, so span attributes hold only hand-written operation labels, table names, and counts.

updated labels

2026.07.18.1

2026.07.18.1

Changed: Version bump only, no code changes.

2026.07.16.1

Changed: Internal-only version bump. PR #183 added a fmt:check task to deno.json, which is not part of the published bundle — this release's published content is identical to 2026.06.30.1.

2026.07.16.1

2026.07.16.1

Changed: Internal-only version bump. PR #183 added a fmt:check task to deno.json, which is not part of the published bundle — this release's published content is identical to 2026.06.30.1.

2026.06.30.1

2026.06.30.1

Two-Phase Datastore Sync

Adds preparePush/commitPush two-phase sync protocol alongside the existing fullWalkPush and pushOneRel single-shot paths.

Added

  • TwoPhaseSyncService interface with preparePush() and commitPush() methods
  • PushManifest opaque branded type for safe manifest passing between phases
  • preparePush() captures sidecar snapshot and collects diff without transaction
  • commitPush() executes batched inserts in a transaction, clears sidecar
  • capabilities() now reports twoPhaseSync: true

Changed

  • Extracted collectFullWalkDiff() and collectOneRelDiff() helper functions
  • Refactored fullWalkPush/pushOneRel to use extracted helpers (behavior preserved)
  • No breaking changes to existing sync behavior
2026.06.15.1
2026.06.11.1
2026.05.30.1
2026.05.25.1
05Stats
A
100 / 100
Downloads
69
Archive size
74.8 KB
  • Has README or module doc2/2earned
  • README has a code example1/1earned
  • README is substantive1/1earned
  • Most symbols documented1/1earned
  • No slow types (deprecated)1/1earned
  • Dependencies pass trust audit2/2earned
  • Has description1/1earned
  • Platform support declared (or universal)2/2earned
  • License declared1/1earned
  • Verified public repository2/2earned
06Platforms
07Labels