Relationships
#2003 buildDeps should reuse repoContext.unifiedDataRepo instead of creating a standalone repo per step
Opened by magistr · 9/4/2026
Problem
Every workflow step on serve creates a new FileSystemUnifiedDataRepository in buildDeps (execution_service.ts). This standalone repo:
- Has no hydrateFile hook — fixed partially by #1984, but the per-step pullChanged sync still runs to compensate
- Triggers a full pullChanged via the stepLockHook before each step, adding ~60s of MongoDB overhead even when nothing changed remotely
- Creates a new MongoDB connection per step (no connection pooling across the sync service instances)
After the MongoDB datastore migration, this turned 1-5s workflows into 60-350s ones, forcing all 19 scheduled workflows to be disabled.
Proposed fix
Pass repoContext.unifiedDataRepo (which already has hydrateFile, markDirty, and a live catalog) into the step's method context instead of creating a standalone repo. The serve process's repo is the authoritative local cache — steps reading from it see the latest data without any sync.
The stepLockHook would still acquire the model lock but skip pullChanged when the step uses the serve's own repo (since it's already up to date from the serve-level sync).
Evidence
| Workflow | Pre-migration | Post-migration | Schedule |
|---|---|---|---|
| fleet-health | 1-5s | 62-195s | */2 min (disabled) |
| swamp-watch-serve | 8-50s | 337-560s | */5 min (disabled) |
| anilist-activity-hourly | ~100s | ~210s | hourly |
All 19 scheduled workflows disabled 2026-09-04 pending this fix. Datastore-side mitigations (watermark probe #222, debounce, sole-writer check) cut 195s→62s but can't eliminate the MongoDB connection overhead.
Related: #1984 (hydrateFile wiring), #1991 (data delete silently reverts)
Closed
No activity in this phase yet.
stack72 commented 9/4/2026, 5:26:08 PM
Investigation found the root cause is in the MongoDB datastore extension, not swamp core.
Root cause: sync.ts in the MongoDB extension has a watermark fast-path probe for pullChanged, but it only fires for unscoped pulls (if (!scoped && ...)). Since the extension advertises scopedSync: true, core passes context.models on every per-step lock acquisition, making every pull scoped — and the fast path never fires. Every scoped pull then walks all path docs under the model prefix, reads every local file, SHA-256 hashes it, and compares against the remote. That's the ~60s per step.
The S3 and GCS datastore extensions don't have this problem — their tryFastPullChanged() runs before the scoped/unscoped branch and fires for all pulls regardless of scope.
Fix: Add a scoped watermark probe to the MongoDB extension's pull() function — one indexed findOne query checking if any doc under the model prefix changed since lastPulledAt. Sub-second, and skips the entire file scan when nothing changed.
Filed upstream: keeb/swamp-mongodb-datastore#11 (umag/swamp-mongodb-datastore has issues disabled)
Sign in to post a ripple.