Relationships
#671 Telemetry flush blocks command exit, adding ~1.2s p50 to every invocation
Opened by magistr · 6/17/2026· Shipped 6/17/2026
Summary
Every CLI invocation awaits a synchronous network flush of usage telemetry before exiting, adding ~1.2 s (p50) of wall-time to every command. The telemetry event is already persisted to the local spool before the flush, so the network push can be deferred/detached without any data loss.
(This is the usage-telemetry HTTP flush, not OpenTelemetry — repro is with no OTEL_EXPORTER_OTLP_ENDPOINT set, so the OTel SDK is never loaded.)
Impact
~1.2 s added to every command that records telemetry (i.e. all real commands; --version is exempt because it exits before the teardown flush). On a ~250-model / ~0.9 GB repo on macOS/Intel this is 35–45 % of total command wall-time.
Evidence (A/B, same shipped binary, median of 7 runs)
| command | telemetry ON | SWAMP_NO_TELEMETRY=1 |
delta |
|---|---|---|---|
config list |
3050 ms | 1798 ms | −1252 ms |
model search --json |
3274 ms | 2116 ms | −1158 ms |
--version (control) |
1685 ms | 1622 ms | −63 ms |
The --version control barely moves (it skips the flush), isolating the delta to the telemetry flush. Telemetry-ON runs also carry a long, variable tail (3.8–4.1 s) — the network round-trip jitter sitting on the critical path. Disabling telemetry removes the cost but also stops event reporting, which isn't an acceptable trade for users who want both speed and reporting.
Root cause
src/cli/mod.ts teardown awaits the flush on the success path (~line 1290) and again on the error path (~line 1417):
await telemetryCtx.service.recordSuccess(commandInfo, startTime); // persists event to spool
...
await telemetryCtx.service.flushTelemetry({ sender, ..., signal: AbortSignal.timeout(2000) });recordSuccess() already writes the entry durably to {repo}/.swamp/telemetry/telemetry-*.json before the flush, and the existing code comment notes unsent data "will be flushed on the next invocation." So the awaited network POST is the only thing on the critical path, and it is not load-bearing for durability.
Proposed direction
Make the flush non-blocking. Because events are already spooled, the POST can be handed off — a detached short-lived flusher, a background per-repo daemon, or a sendBeacon-style fire-and-forget — so the command exits immediately and the spool drains out-of-band (with the existing "flush on next invocation" path as the safety net).
POC (verified locally against current main)
Two small edits in src/cli/mod.ts: (1) a top-of-runCli short-circuit that, when invoked as a detached child, performs one blocking flushTelemetry() and returns — recording no event and never re-spawning (no loop); (2) the success teardown keeps recordSuccess() but replaces await flushTelemetry(...) with a detached, unref()-ed child and no await.
Results (from source, median of 5):
| command | unpatched (blocking) | patched (async) | speedup |
|---|---|---|---|
config list |
3371 ms | 1871 ms | 1.80× |
model search --json |
3027 ms | 2054 ms | 1.47× |
Foreground latency drops to the telemetry-disabled floor while still delivering events — the local spool was observed draining from 1 → 0 within a few seconds after each command, confirming the detached flush completes the network POST. The foreground jitter tail is also eliminated (patched runs are tight, ±~80 ms). deno check main.ts is clean.
Caveat: in the POC the detached child re-pays Deno cold-start (~1.7 s) in the background per command; a production fix would use a lightweight flusher (sender + spool only) or a persistent daemon rather than a full re-exec.
Environment
- swamp
20260610(shipped binary) and currentmainsource — blocking flush present in both - macOS, Intel x86_64
- Default config; no OTLP endpoint configured
Shipped
Click a lifecycle step above to view its details.