Skip to main content
← Back to list
01Issue
BugIn ProgressSwamp CLI
Assigneesstack72

Relationships

#1310 serve: workflow.approvals and workflow.run.search still full-scan the workflow-run store after the #1284 data.query pushdown

Opened by ehazlett · 7/21/2026

Summary

The read-scan fix from Lab #1284 landed for data.query — its zero-row floor is gone and predicate matches are now fast. But the sibling workflow.* read ops, workflow.approvals and workflow.run.search, were not covered: they still linearly scan every workflow run and CEL-filter after, so they pay a ~500ms floor regardless of how few rows match. Any client that polls workflow.approvals eats a full scan on every cycle, and the cost grows with the total number of runs rather than with the number of pending approvals.

Relationship to #1284

#1284 reported data.query, workflow.approvals, and workflow.run.search all scanning. The shipped fix resolved data.query (confirmed below) but left the two workflow.* ops unchanged — this issue tracks that remaining half.

Environment

  • swamp 20260720.234931.0-sha.4770b0fd
  • serve with token auth, single repo, ~600 total data records, 38 workflow runs (1 suspended at a gate).

Measurements

Server-side cost, isolated by subtracting a constant client/transport baseline; each op run several times, median reported.

op rows returned before (#1284, swamp 20260718) after (swamp 20260720)
data.query, zero-row predicate 0 ~346ms ~0ms (fixed)
data.query, match-all ~600 ~490ms ~130ms (fixed)
workflow.run.search, all runs 38 ~486ms (37 runs) ~545ms (unchanged)
workflow.approvals 1 ~500ms ~528ms (unchanged)

Why this reads as an unfixed scan

workflow.approvals returns 1 row but costs the same (~528ms) as workflow.run.search returning all 38 (~545ms). The cost tracks rows scanned, not rows returned — the same "load and evaluate every record, then filter" signature #1284 described for data.query, on a store the pushdown did not reach. As the run store grows, both ops get slower even when the pending-approval count stays at one.

Impact

A client that lists pending approvals must either poll workflow.approvals and pay a full scan per cycle, or front it with a client-side cache. The floor scales with total run history, so it degrades over time.

Suggested direction

Extend the #1284 predicate pushdown / index to the workflow-run store: an index for the common shapes (run status such as suspended, workflow name), and short-circuit cheap top-level equality before materializing full run bodies — so workflow.approvals costs proportional to pending gates, not to total runs.

Repro

  1. Populate a repo with a few dozen workflow runs, one suspended at a manual-approval gate.
  2. Over a serve connection, time workflow.approvals (returns the 1 suspended run) and workflow.run.search (returns all runs).
  3. Observe both cost ~500ms and are roughly equal despite very different result sizes; compare against a fast data.query on the same connection to confirm the floor is specific to the workflow.* read path.
  • Lab #1284 (the data.query half — shipped)
  • Lab #1285 (idle serve event-loop CPU — aggravates the tail latencies here, separate root cause)
02Bog Flow
OPENTRIAGEDIN PROGRESSSHIPPED+ 1 MOREASSIGNED+ 5 MOREREVIEW+ 3 MOREPR_LINKED

In Progress

7/21/2026, 4:44:08 PM

Click a lifecycle step above to view its details.

03Sludge Pulse
stack72 assigned stack727/21/2026, 3:36:21 PM

Sign in to post a ripple.