Skip to main content
← Back to list
01Issue
BugShippedSwamp CLIPublic
Assigneesstack72

Relationships

#1898 Namespace mismatch regression in datastore sync after PR #2314 lock key restructuring

Opened by stack72 · 8/28/2026· Shipped 8/28/2026

Description

PR #2314 (fix(core): namespace-scope datastore lock key and pass namespace to all pushChanged sites) introduced a regression where all serve-mode datastore sync operations fail with:

Datastore sync failed: could not pull data for command/shell/<id>: Namespace mismatch: bound to undefined but called with "<namespace>"

The datastore instance is created/bound without a namespace (undefined), but the new pushChanged() call sites now pass a namespace, triggering the mismatch guard.

Steps to Reproduce

  1. Build swamp at commit 6cc72797 (version 20260828.215434.0-sha.6cc72797)
  2. Start a serve instance with an S3/MinIO datastore
  3. Run any workflow via swamp workflow run <name> --server <url>
  4. Observe the workflow fails with the namespace mismatch error

Impact

This breaks:

  • Workflow execution in serve mode (all workflow runs fail)
  • Grant replication across HA cluster instances (grants don't propagate)
  • Datastore hydration from pre-populated buckets
  • Namespace isolation between servers

Evidence

8 UAT tests fail across 2 CI jobs:

Access Grants job (2 failures):

  • swamp serve cluster replicates grants across instances via reloads (ha_test.ts:91)
  • swamp serve cluster access data poller propagates grants without a reload (ha_test.ts:196)

OAuth + Datastore + HA job (6 failures):

  • swamp serve runs a workflow with outputs persisted to the bucket (datastore_test.ts:180)
  • swamp serve hydrates the cache from a pre-populated bucket (datastore_test.ts:232)
  • swamp serve namespaces do not leak between servers (datastore_test.ts:324)
  • swamp serve rolling restart preserves running workflows (ha_operations_test.ts)
  • swamp serve data operations work across HA instances (ha_operations_test.ts)
  • swamp serve workflow outputs are visible in the S3 bucket (ha_operations_test.ts:218)

CI run: https://github.com/swamp-club/swamp-uat/actions/runs/33214676379

Root Cause

PR #2314 changed datastoreGlobalLockOptions to return { lockKey: ".datastore.lock", namespace } instead of baking namespace into the key path. It also added namespace to 14 pushChanged() call sites. However, the datastore instances used in serve mode appear to be bound without a namespace, so when the new code passes a namespace to operations on those instances, the namespace mismatch guard fires.

02Bog Flow
✓OPEN✓TRIAGED✓IN PROGRESS✓SHIPPED+ 1 MOREASSIGNED+ 5 MOREREVIEW+ 7 MOREPR_MERGED+ 2 MORESESSION_SUMMARIZED

Shipped

8/28/2026, 11:18:02 PM

Click a lifecycle step above to view its details.

03Sludge Pulse
stack72 assigned stack728/28/2026, 10:17:32 PM
Editable. Press Enter to edit.

stack72 commented 8/28/2026, 10:38:16 PM

Debugging context for the fix

The error is Namespace mismatch: bound to undefined but called with "<namespace>". This tells us the datastore's sync service was constructed without a namespace binding, but pushChanged() is now being called with a namespace object { namespace }.

Where the bug is

The PR changed pushChanged() from a no-arg call to pushChanged({ namespace }) at 14 call sites. The namespace value comes from:

const namespace = isCustomDatastoreConfig(datastoreConfig)
  ? datastoreConfig.namespace
  : undefined;
await syncService.pushChanged({ namespace });

But the syncService itself was created earlier in the lifecycle (during repo context initialization or serve startup) and its internal datastore was bound without namespace awareness. When pushChanged receives a namespace and forwards it to the underlying datastore pull/push operations, those operations check the namespace against what the datastore was bound with — and it's undefined, hence the mismatch.

The specific error path

The error message could not pull data for command/shell/<id>: Namespace mismatch: bound to undefined but called with "<namespace>" indicates the mismatch check is inside the datastore's pull method (or equivalent), not in the lock layer. The lock changes (FileLock, datastoreGlobalLockOptions) look correct — they properly scope locks under {namespace}/. The problem is that pushChanged is passing a namespace to a datastore that wasn't initialized with one.

Likely fix

Either:

  1. Bind the namespace when constructing the sync service — so the datastore knows its namespace from creation and the pushChanged({ namespace }) calls match.
  2. Have pushChanged derive the namespace internally from its already-bound datastore config rather than accepting it as a parameter — this would avoid the mismatch by construction.

Option 2 is probably safer since it means callers can't accidentally pass a wrong/mismatched namespace.

Files to check

  • Wherever DatastoreSyncService is constructed (likely repo_context.ts and src/serve/deps.ts) — is namespace passed to the underlying datastore at construction time?
  • The datastore's pull/push implementation that performs the Namespace mismatch guard — what does "bound to" mean in its context? That's where the expected namespace is set.
  • executeWorkflowWithLocks in src/serve/deps.ts — this is the serve-mode workflow execution path that fails.

Sign in to post a ripple.