Relationships
#1620 Model lookup misreports EMFILE (too many open files) as a broken definition file and dumps a raw stacktrace
Opened by keeb · 8/12/2026· Shipped 8/12/2026
Reported by a user running a workflow against a repo with a large model tree.
What happens
When the process hits the open-file-descriptor limit during a global model lookup, Deno.readTextFile throws Error: Too many open files (os error 24) with code: "EMFILE". That error is thrown inside the try block at src/infrastructure/persistence/yaml_definition_repository.ts:224, which is wrapped by a catch (parseError) intended to tolerate malformed YAML:
} catch (parseError) {
logger.warn`Skipping broken definition file ${fullPath}: ${parseError}`;
}Two problems follow:
Misclassification. A resource-exhaustion error is reported as
Skipping broken definition file. The definition is not broken — it was never read.searchDefinitionByNamethen continues the walk and can returnnull, sofindByNameGlobal→findDefinitionByIdOrName→DefaultStepExecutor.executeModelMethodbehaves as though the model does not exist, rather than failing on an I/O fault. Under FD pressure this is nondeterministic: the same workflow can resolve a model on one run and not on the next.Raw stacktrace as output. The error is interpolated into the log line as
${parseError}, so the user sees a full multi-frame async stacktrace (including internalext:deno_fsframes and compiled-binary temp paths) atwarninglevel, with no statement of what went wrong or what to do about it.
Expected
- I/O and resource-exhaustion errors (
EMFILE,ENFILE,EACCES,EIO, etc.) should be distinguished from YAML parse/validation errors. Only the latter belong in the "skip this file and keep going" path. - An FD-exhaustion error should abort the lookup and surface as a structured swamp error that fails the workflow step with an actionable message — e.g. that the open-file limit was reached while scanning model definitions, and that
ulimit -nmay need raising — instead of degrading into a "model not found" outcome. - No raw stacktrace in normal output; the stacktrace belongs behind debug logging.
Reproduce
- In a repo with a deep model tree (many
@namespace/type/directories, each containing many.yamldefinitions), lower the descriptor limit:ulimit -n 64. - Run a workflow whose step resolves a model by name, forcing a global lookup.
- Observe
warning definition-repo Skipping broken definition file ...carrying anEMFILEstacktrace, and the step failing for the wrong reason.
Note that searchDefinitionByName recurses with a Deno.readDir iterator open at every level of the walk, so descriptor pressure scales with directory depth — worth checking alongside the error-handling fix.
Environment
- macOS, compiled
swampbinary - Occurred during
swamp workflow run, in the model-lookup path of a step
Related but distinct from the previously shipped serve-side FD leaks (#1118, #1120); this is the definition-repository read path, not serve.
Shipped
Click a lifecycle step above to view its details.
Sign in to post a ripple.