Relationships
#1120 serve: create*Deps() factories leak file-based catalog stores (3 FDs) per API request
Opened by keeb · 7/13/2026· Shipped 7/13/2026
Summary
Several create*Deps() factory functions open a file-based SQLite
CatalogStore (via createCatalogStore) with no way to close it. In one-shot
CLI commands this is harmless (the process exits and the OS reclaims the FDs),
but under swamp serve these factories are invoked per API request, so every
request against an affected endpoint permanently leaks file descriptors. A
file-based SQLite store in WAL mode holds 3 FDs (.db + -wal + -shm),
so each leak burns 3 descriptors.
This was discovered while investigating #1118 (the workflow log-sink FD leak) and is filed separately per the "keep the PR focused" scope decision. It is an independent leak with a different mechanism.
Mechanism
createCatalogStore(repoDir, datastoreResolver) returns a new CatalogStore(...)
that opens a SQLite connection. Some create*Deps factories only call it when no
shared repo is injected — e.g. createDataListDeps skips it when the serve
handler passes [HOST-1]. But others call it
unconditionally and never expose a close(), e.g.:
createModelEvaluateDeps—src/libswamp/models/evaluate.ts:99openscreateCatalogStore(...)unconditionally; the serve handlermodel_handlers.ts:1078calls it permodel.evaluaterequest and never closes the store.
The general anti-pattern: any serve handler that calls a create*Deps(ctx.repoDir, ...)
factory without injecting the shared ctx.repoContext repo, where that
factory internally opens createCatalogStore, leaks 3 FDs per request. The
cli/commands/datastore_* call sites correctly pair createCatalogStore with
.close(); the libswamp create*Deps factories used by serve handlers do not.
Impact
A serve instance that is polled on an affected endpoint (dashboard, monitoring
UI, scripted clients) climbs toward its RLIMIT_NOFILE soft limit (default 1024)
and eventually fails every Deno.open/Deno.readDir with
Too many open files (os error 24) — the same end state as #1118, via a
different leak.
Empirical note (from the #1118 investigation)
- A file-based SQLite DB in WAL mode costs 3 OS FDs each (verified: 100 leaked file DBs → +300 FDs).
- An in-memory (
:memory:) SQLite DB costs 0 OS FDs — so ephemeral stores are not implicated here; only the file-basedcreateCatalogStorepath is.
Steps to reproduce
- Start
swamp serve. - Repeatedly call an endpoint whose handler builds deps via a
create*Depsfactory that openscreateCatalogStorewithout an injected shared repo (e.g.model.evaluate). - Watch
ls -l /proc/<pid>/fd— 3 new_catalog.db/-wal/-shmhandles accumulate per request and are never released.
Suggested fix
Audit every create*Deps factory that calls createCatalogStore, and for each
serve handler that invokes one:
- Give the factory a shared-repo escape hatch (accept an injected
UnifiedDataRepository/CatalogStoreand skipcreateCatalogStorewhen provided) — mirroring howcreateDataListDepsalready does it — and have serve handlers passctx.repoContext, or - Return a
cleanup()/close()from the factory (or the deps object) and have the handler call it in afinally.
Prefer the shared-repo injection so serve reuses the one long-lived catalog connection rather than opening and closing one per request.
Environment
swamp serve, Linux, Deno runtime.RLIMIT_NOFILEsoft=1024 by default.- Related: #1118 (workflow log-sink FD leak — separate mechanism, same end state).
Shipped
Click a lifecycle step above to view its details.
Sign in to post a ripple.