Relationships
#424 swamp serve: extension bundle failure at startup is silently pinned, breaking scheduled workflows with permanent 'Unknown model type'
Opened by keeb · 5/23/2026· Shipped 5/23/2026
Summary
When swamp serve's eager extension load at daemon startup fails to bundle an extension, the failure is swallowed, recorded as a sticky BundleBuildFailed catalog entry, and never retried — across daemon restarts and even across later successful online runs. Scheduled (and WebSocket) workflow execution then fails permanently with Unknown model type: <type>, while CLI swamp workflow run can still resolve the type if its bundle was warmed first. This makes a freshly-deployed swamp serve appear to "lose" a model type that the CLI resolves fine on the same machine, repo, user, and binary.
Environment
- swamp
20260521.221703.0-sha.3f2b75f8(also reproduced deterministically on currentmain,8ba74414) - Linux, systemd-managed
swamp serverunning as root (vanilla unit:WorkingDirectory,ExecStart=swamp serve --repo-dir ...,Environment=NO_COLOR=1; no resource limits or sandboxing) - A model extension importing an npm dependency (e.g.
npm:nodemailer), a cold deno cache, and no/blocked outbound network to npmjs.org at first bundle
Steps to reproduce
- Fresh repo. Add a model extension importing an npm dep, e.g.
import nodemailer from "npm:nodemailer@6.9.16";, type@user/foo/bar. - Ensure no prior bundles exist:
.swamp/bundles/and.swamp/_extension_catalog.dbabsent (fresh deploy, before any interactive CLI run bundled anything). - Create a model definition and a workflow with
trigger.schedule: "* * * * *"invoking a method on it. - Start
swamp servewith a cold deno cache and no outbound network to registry.npmjs.org (e.g. a freshly-provisioned VM whose npm/deno cache was never warmed and/or whose service has blocked egress). - Wait for the cron to fire.
Expected
Either the extension bundles (when network is available) and the type resolves, OR the daemon surfaces a loud, actionable error that the extension could not be bundled — and retries when conditions change (network restored, restart).
Actual
deno bundlefails fetching npm deps from registry.npmjs.org (tcp connect error: Network is unreachable).- The failure is swallowed; the catalog records
BundleBuildFailedwith an emptytype_normalized; the type is excluded from registration. - The scheduled run throws
Unknown model type: @user/foo/bar. - The failure is sticky: re-bundling only happens on a source fingerprint mismatch, so a
BundleBuildFailedrow whose fingerprint still matches is never retried. Restarting the service does not clear it, and even a later online CLI run on the same repo does not recover it.
Why the CLI "works" but serve doesn't
Ordering. Whoever bundles first while online wins. If an interactive CLI warmed .swamp/bundles/ before serve started, serve reuses the cache and resolves fine. On a fresh deploy where serve starts first on a cold cache with no egress, the failure is pinned — the operator then sees the CLI succeed (its run used/warmed good bundles) while the long-running daemon stays broken across restarts.
Root cause (code)
src/domain/models/bundle.ts:387-406—bundleExtensionspawnsdeno bundle, which must fetch npm deps from the network when the deno cache is cold; no offline /--cached-onlyguard.src/serve/deps.ts:66-71— serve does a one-shot eagermodelRegistry.ensureLoaded()at daemon startup, bundling the whole extension suite.src/domain/extensions/extension_loader.ts:342-373and:704-713— bundle failures are caught, recorded asBundleBuildFailed, and excluded from lazy registration.src/domain/extensions/bundle_freshness.ts:307-321—findStaleFilesonly re-bundles on a fingerprint mismatch, so a fingerprint-matching failed entry is pinned across restarts and later online runs.src/domain/workflows/execution_service.ts:469—resolveModelTypereturns undefined and throwsUnknown model type.
Suggested fixes
- Do not treat
BundleBuildFailedas a satisfied fingerprint-cache entry — retry failed builds (on restart at minimum). - Fail loud at serve startup when an extension cannot be bundled, instead of silently dropping the type.
- Offline/cold-cache handling: pre-warm bundles, support
--cached-onlysemantics, or surface a clear "could not fetch npm deps to bundle X" error to the operator.
Shipped
Click a lifecycle step above to view its details.
Sign in to post a ripple.