#3741 serve: add child spans and attributes to workflow.history.get (and run_summary/run_search) to diagnose slow dashboard loads

Problem

The workflow dashboard on the ops server (swamp-serve-vm) loads slowly, and the traces don't show why. swamp.workflow.history.get is emitted as a root span with no child spans and no attributes, so a slow call can't be broken down into datastore read, lock wait, record parsing or serialization.

What the traces show (Axiom swamp-serve-traces, last 7d)

swamp.workflow.history.get latency by serve build:

• Calls arrive in bursts of 8–17 within 10s, and all calls in a burst take the same time. For example, on 2026-10-21 between 17:04 and 17:06 UTC each call took about 83s. Calls ending together point to queueing on one shared resource (a lock, a single scan, or a blocked event loop), but the current spans can't confirm it.
• In that 8-minute window there were 239 history.get spans, totaling about 9,500 span-seconds, with essentially one human viewer. That suggests the dashboard fans out one request per workflow, or polls.
• At similar load, the hourly p50 rose from about 0.6s (Oct 19) to between 2.5s and 4.6s (Oct 21). This could be history growth or a code regression; the traces can't tell them apart.
• Slow windows line up with heavy UAT data-plane traffic (POST /data/* p95 between 30s and 77s, and swamp.lock.acquire p95 peaks of 14s and 29s) and with swamp.retention.batch, which first appears on Oct 19 at the same time history.get volume jumped.
• swamp.workflow.run_summary, run_search and approvals show the same pattern: a p99 between 10s and 32s, and they are also leaf spans with no attributes.

Related, already shipped: #3407 (search parses every run record) and #3444 (the dashboard polls workflow.run.search).

Proposed tracing

On swamp.workflow.history.get, and also on run_summary, run_search and approvals:

Attributes

• swamp.workflow.name, swamp.workflow.run_id, and whether the lookup was by name, id or prefix
• swamp.history.runs_scanned, swamp.history.runs_returned, swamp.history.bytes_read
• swamp.datastore.kind (v3, s3, local, and so on)
• the caller's principal kind (user, worker, token), to separate dashboard fan-out from other callers
• the queue wait between receiving the request and starting the handler, if the server serializes requests

Child spans

• datastore listing and index lookup
• lock acquisition, if any (or link to the existing swamp.lock.acquire span as a child)
• per-record read and parse, or one span with an aggregate count
• authorization filtering
• response serialization

Request context

• Parent these spans under the inbound WebSocket or HTTP request span, so one dashboard page load appears as a single trace and the fan-out is visible.

Optional

• An event-loop lag metric or span event on serve, to confirm or rule out event-loop blocking during bursts.

Why

With these, one slow trace would show whether the time goes to lock contention with UAT data-plane writes or retention, to history growth (runs scanned), to a code regression between builds 20261015 and 20261021, or to dashboard fan-out.