← Back to list
01Issue
BugShippedSwamp CLIPublic
Assigneesstack72
Relationships
#4580 Telemetry still adds about 0.9 s to every command on the dogfood VMs despite the detached sender (F33, follows #4511)
Opened by stack72 · 11/10/2026· Shipped 11/10/2026
Follows swamp-club#4511 (PR #3670), which moved telemetry sending into a detached sender process so commands don't wait on the ingest. On the team-mode dogfood VMs (swamp-club#4374) that saving hasn't shown up: commands with telemetry on still take about 0.9–1.0 s longer than with it off.
Evidence
| Run | Release | swamp version, telemetry on minus off |
|---|---|---|
| 5 (part B) | 20261108.111402.0-sha.37c27525 (with #4511) |
+893 ms (bar: 50 ms) |
| 6 (part B) | 20261109.214205.0-sha.2c24fe25 |
+0.96 s |
- Start-up itself is fine: with telemetry off it's back at run 3's level (1.73 s on run 6 against 1.82 s on run 3), so the extra cost is telemetry alone.
- #4511's local measurement on Linux arm64 (1 CPU, commands back to back) showed telemetry on within about 40 ms of off, and one sender per burst of 10.
- The dogfood VMs are small Linux x86_64 GCE instances. Each member's
swamp versionwas run 10 times with telemetry on and 10 times off.
Work
- Find what still waits on the VMs. Trace one
swamp versionwith telemetry on and off on a dogfood member (the #4374 session can run it), with spans orstrace -f -ttacross the command's exit. Candidates:- the sender's spawn (re-exec of the binary) still awaited, or blocking on a pipe or handle;
- the spool write's fsync;
- reading auth or config at the end of the command;
- the 1 s exit drain from #4511 still being waited out (for example, an OTel exporter or log sink left open when telemetry is on);
- the lock-file dance on the sender lock.
- Remove the wait.
Tests
- A test where the telemetry ingest never answers and the sender is slow to start: the command's exit isn't delayed (assert on work done or process state, not wall-clock bounds).
- #4511's tests keep passing.
Done when
On the dogfood VMs, swamp version with telemetry on is within 50 ms of off (10 runs each).
02Bog Flow
Shipped
Click a lifecycle step above to view its details.
03Sludge Pulse
Sign in to post a ripple.