Skip to main content
← Back to list
01Issue
FeatureClosedSwamp CLIPublicTeam
Assigneesstack72

Relationships

#2003 buildDeps should reuse repoContext.unifiedDataRepo instead of creating a standalone repo per step

Opened by magistr · 9/4/2026

Problem

Every workflow step on serve creates a new FileSystemUnifiedDataRepository in buildDeps (execution_service.ts). This standalone repo:

  1. Has no hydrateFile hook — fixed partially by #1984, but the per-step pullChanged sync still runs to compensate
  2. Triggers a full pullChanged via the stepLockHook before each step, adding ~60s of MongoDB overhead even when nothing changed remotely
  3. Creates a new MongoDB connection per step (no connection pooling across the sync service instances)

After the MongoDB datastore migration, this turned 1-5s workflows into 60-350s ones, forcing all 19 scheduled workflows to be disabled.

Proposed fix

Pass repoContext.unifiedDataRepo (which already has hydrateFile, markDirty, and a live catalog) into the step's method context instead of creating a standalone repo. The serve process's repo is the authoritative local cache — steps reading from it see the latest data without any sync.

The stepLockHook would still acquire the model lock but skip pullChanged when the step uses the serve's own repo (since it's already up to date from the serve-level sync).

Evidence

Workflow Pre-migration Post-migration Schedule
fleet-health 1-5s 62-195s */2 min (disabled)
swamp-watch-serve 8-50s 337-560s */5 min (disabled)
anilist-activity-hourly ~100s ~210s hourly

All 19 scheduled workflows disabled 2026-09-04 pending this fix. Datastore-side mitigations (watermark probe #222, debounce, sole-writer check) cut 195s→62s but can't eliminate the MongoDB connection overhead.

Related: #1984 (hydrateFile wiring), #1991 (data delete silently reverts)

02Bog Flow
✓OPEN✓TRIAGED○IN PROGRESS◉CLOSED+ 1 MOREASSIGNED+ 2 MOREREVIEW

Closed

9/4/2026, 5:26:09 PM

No activity in this phase yet.

03Sludge Pulse
stack72 assigned stack729/4/2026, 4:53:12 PM
Editable. Press Enter to edit.

stack72 commented 9/4/2026, 5:26:08 PM

Investigation found the root cause is in the MongoDB datastore extension, not swamp core.

Root cause: sync.ts in the MongoDB extension has a watermark fast-path probe for pullChanged, but it only fires for unscoped pulls (if (!scoped && ...)). Since the extension advertises scopedSync: true, core passes context.models on every per-step lock acquisition, making every pull scoped — and the fast path never fires. Every scoped pull then walks all path docs under the model prefix, reads every local file, SHA-256 hashes it, and compares against the remote. That's the ~60s per step.

The S3 and GCS datastore extensions don't have this problem — their tryFastPullChanged() runs before the scoped/unscoped branch and fires for all pulls regardless of scope.

Fix: Add a scoped watermark probe to the MongoDB extension's pull() function — one indexed findOne query checking if any doc under the model prefix changed since lastPulledAt. Sub-second, and skips the entire file scan when nothing changed.

Filed upstream: keeb/swamp-mongodb-datastore#11 (umag/swamp-mongodb-datastore has issues disabled)

Sign in to post a ripple.