Skip to main content
← Back to list
01Issue
FeatureClosedSwamp ClubPublic
AssigneesNone

Relationships

#2883 Docs: document serve and worker trace structure in the OpenTelemetry reference

Opened by stack72 · 10/1/2026

The manual's content/manual/reference/opentelemetry.md has no section on how swamp serve and swamp worker traces are structured, and its CLI Root Span section implies every span sits under swamp.cli. swamp-club#2467 (PR swamp-club/swamp#2763) changed this.

Suggested content for a new 'Traces in swamp serve and workers' section:

  • Root spans for background work. HTTP requests are SERVER root spans. Each poll cycle is a root swamp.serve.poll span. Its swamp.serve.poller attribute is config, access_data, runtime_data or grants_directory, and its sync-gate, datastore sync and datastore extension spans nest under it. Each scheduled fire is a root swamp.scheduled.fire span with the workflow run under it. A WebSocket or queued webhook run without traceparent starts its own trace. Other loops (heartbeats, GC, audit flushes, token revalidation) emit no per-tick spans.
  • Worker dispatches. A dispatch joins the run's trace when serve sends trace headers: swamp.remote.dispatch sits under the step's swamp.model.method, then the runner's swamp.cli. swamp.serve.data_plane spans, with a swamp.data_plane.operation attribute, record the runner's data-plane reads and writes in the run's trace.
  • Runner environment. Runners never inherit TRACEPARENT from the worker's environment. Serve no longer ships its OTEL_* settings or trace context to workers, so runners export with the worker host's own OTel settings. Configure OTEL_* on each worker host.
  • Console exporter. Console span output from a runner goes to stderr and appears in the worker log at debug level.
  • Export at exit. A process waits up to the 10s exporter timeout for in-flight span exports when it exits, including on the error path.
  • Correction. The Coverage section's statement that worker subprocesses receive trace context via TRACEPARENT is still true, but only from the dispatch's own trace headers.
02Bog Flow
✓OPEN○TRIAGED○IN PROGRESS◉CLOSED

Closed

10/2/2026, 1:46:21 AM

No activity in this phase yet.

03Sludge Pulse
Editable. Press Enter to edit.

stack72 commented 10/2/2026, 1:46:20 AM

Documented in https://github.com/swamp-club/swamp-club/pull/1281. reference/opentelemetry.md has a new section, 'Traces in swamp serve and workers'. It covers root spans for HTTP requests, swamp.serve.poll cycles (with the swamp.serve.poller values), scheduled fires and webhook runs, and notes that heartbeat, GC and similar loops emit no per-tick spans. It also covers how worker dispatches nest (swamp.remote.dispatch, and swamp.serve.data_plane with swamp.data_plane.operation), that runners never inherit TRACEPARENT and need OTEL_* set on each worker host, console exporter output going to stderr, and the 10s export wait at exit. The CLI Root Span and Coverage sections now point to it, and the webhook paragraph notes that a run without traceparent starts its own trace. Span names were checked against swamp source.

Sign in to post a ripple.