Relationships
#2556 InProcessExecutor mutates the process-wide TRACEPARENT, so concurrent in-process executions clobber and leak trace context
Opened by hammz · 9/25/2026· Shipped 9/28/2026
Problem
InProcessExecutor.execute (src/domain/models/in_process_executor.ts, around lines 285-372) writes the request's traceHeaders into the process-wide Deno.env (TRACEPARENT, TRACESTATE) before running the method, and in a finally restores the values it snapshotted on entry, deleting the variable when the snapshot was unset.
Deno.env is shared by every concurrent execution in the process. Workflow steps in the same level run concurrently in-process (execution_service.ts honours workflow/job/step concurrency), and swamp serve runs many executions in one process. With tracing enabled:
- Step A snapshots unset, sets TRACEPARENT to A's span.
- Step B snapshots A's value, sets TRACEPARENT to B's span.
- A finishes and deletes TRACEPARENT while B is still running, so any subprocess B spawns loses its trace context.
- B finishes and restores A's value, which then leaks into the process after both steps are done, so later subprocesses are parented to a finished span.
The same save/restore also races with tests across parallel test files (deno test --parallel shares one process env across files); it was the mechanism behind the flake in swamp-club#2445, and src/cli/context_test.ts and src/domain/models/in_process_executor_test.ts still read and write TRACEPARENT in the process env.
Expected
Per-execution trace context reaches the method (and any subprocess it spawns) without mutating process-global state. For example: carry the trace headers on the method context, and have subprocess-spawning code pass TRACEPARENT through the child's env explicitly. The two tests above should then stop mutating Deno.env.
Found while triaging swamp-club#2445.
Shipped
Click a lifecycle step above to view its details.
Sign in to post a ripple.