Skip to main content
← Back to list
01Issue
BugShippedSwamp CLIPublic
Assigneesstack72

Relationships

↔ sibling #4712

#4711 Flaky: command routes test times out when a batch of route workers stalls importing main.ts

Opened by stack72 · 11/13/2026· Shipped 11/13/2026

integration/command_routes_rules_test.ts, test 'command routes: each route loads alone in a fresh module graph with the full tree's schema and error handlers', fails intermittently in verify-build with ': no answer within 120000ms'. The sibling test 'a completion callback's routes in a fresh module graph...' has hit the same deadline once.

Rate: 6 failures out of 45 verify-build runs that reached the test in the local workflow-run logs (about 1 in 7), on 7 unrelated branches since 2026-11-10 (serve, datastore and start fixes). The named route differs every time (quest, dashboard, datastore status, init, model type, vault type, extension pull), so the route is not the cause.

What was found (swamp-club#4707 branch, Deno 2.9.8, macOS arm64):

  • Per-stage logging in integration/command_route_worker.ts shows the stall is in the worker's import of main.ts: in the reproducing run, all 4 workers of one batch (datastore quarantine, compact, gc, claim) received their message but never finished the import, while every other worker loaded in under 4s.
  • No module in main.ts's static import graph has a top-level await, and command modules are outside that closure, so no route code has run yet when it stalls.
  • Loading the routes alone never stalls: 680 worker loads across 4 concurrent processes, slowest 1.4s.
  • It reproduces only with other heavy integration files in the same deno test process: the route test plus access_integration_command, fleet_fan_out_*, serve_upstream* and serve_v3_team* tests, 3 processes at once, stalled in 1 of 3 runs.
  • A bisect of which co-running files trigger it was started but not finished.

Likely a Deno module-loading stall when several Workers import concurrently while other test isolates in the same process are busy. Next steps: finish the bisect, then either avoid the trigger (fewer concurrent workers, or isolate this test) or retry a route whose worker never answers in a fresh worker, so a genuine load failure (onerror or a second stall) still fails.

02Bog Flow
✓OPEN✓TRIAGED✓IN PROGRESS✓SHIPPED+ 2 MOREISSUE_LINKED+ 3 MOREFINDINGS+ 8 MOREPR_LINKED+ 2 MORESESSION_SUMMARIZED

Shipped

11/13/2026, 6:17:19 PM

Click a lifecycle step above to view its details.

03Sludge Pulse
stack72 assigned stack7211/13/2026, 2:43:37 PM
stack72 linked sibling of #471211/13/2026, 2:43:46 PM

Sign in to post a ripple.