#606 Resident/warm worker mode — `model method run` has ~6s fixed per-invocation overhead that rules out latency-sensitive use

Problem

swamp model method run carries a large fixed per-invocation cost (~6s warm, ~9s cold on my machine) that is dominated by the method-execution path, not CLI startup or datastore I/O. For latency-sensitive, fire-frequently use cases this makes a swamp model the wrong tool — even when the model is otherwise a perfect domain fit — and forces a downgrade to host-language glue.

Concrete case: a @mgreten/macos-say-tts model speaks a sentence via macOS say. Wrapped in a Claude Code Stop hook (speak each assistant turn aloud), every turn paid the full invocation cost. The actual TTS synthesis is ~0.7s; the wrapper added ~6-9s. I had to rip swamp out of the hot path and replace it with a persistent FIFO daemon calling say directly to get to ~1s end-to-end. The model is still used for on-demand/audited calls, but the recurring path can no longer be a swamp model. I'd much rather have kept it in swamp.

Benchmark / isolation (swamp 20260521.221703.0-sha.3f2b75f8, macOS arm64)

Measured with /usr/bin/time -p, payload is a no-op (say "x", sub-ms work), so all time below is swamp overhead, not the model's own work:

The ~5s delta between model get (~1.1s) and method run (~6.4s) is the method-execution path specifically — not CLI startup (already paid at ~1s) and not datastore reads. Warm runs are tightly clustered (~6.2-6.6s), so it's a fixed cost, not variance: it looks like a fresh Deno isolate + full extension re-bundle/re-eval on every invocation.

Proposed solution

Anything that amortizes the per-invocation method-execution setup would unlock this class of use case. Options, roughly in order of how much they'd help:

1. A resident/warm worker mode — a long-lived swamp daemon process that keeps the runtime + bundled extensions hot, so method run dispatches to it instead of cold-booting an isolate each time. Even an opt-in swamp serve + swamp model method run --via-daemon would do it.
2. Bundle/eval caching — cache the compiled extension bundle + warm isolate keyed by extension content hash, so repeated invocations skip re-bundling.
3. A lighter "exec" path for models that opt out of state recording — skip the datastore write / lock for fire-and-forget methods (--no-record or a model-declared ephemeral: true). Less impactful than (1)/(2) since the datastore write isn't the bulk of it, but cheap to add and composes.

Alternatives considered

• Keep swamp, accept the latency — not viable for a per-turn hook; 8s of silence after every response is unusable.
• Swap the TTS engine (Piper/Kokoro/mlx-audio) — wouldn't help: the ~0.7s synth was never the problem; the wrapper was. Verified and deferred.
• The FIFO daemon I ended up building — works (~1s) but it's exactly the "resident warm process" pattern from proposal (1), reimplemented by hand in bash outside swamp. That's the tell: the runtime could offer this generically and keep these models inside swamp where they belong.

Why this matters beyond TTS

Any "fire a small swamp method frequently / interactively" pattern hits this: hook-driven notifications, per-event enrichment, tight polling loops, anything on a human-perceptible latency budget. Today the guidance (rightly) is "don't put hot-path work in a swamp model" — but a warm-worker mode would expand the boundary of what swamp can own without forcing host-language escape hatches.