Relationships
#3308 swamp serve lists every wait-key record on each continuation pass once any pending signal has lapsed
Opened by zack · 10/12/2026· Shipped 10/12/2026
Problem
Follow-up to swamp-club#3212, raised by the adversarial review of PR 3021. That PR made swamp serve run the pending-signal sweep with every continuation pass (sweepHeldSignals in src/serve/continuation_sweep_service.ts, default every 30 seconds), so pending records are cleaned up where they gather.
sweepPendingSignals in src/domain/workflows/signal_wait_cleanup.ts returns early only when there are no pending addresses at all. A lapse is never deleted: by design one stays per slot until the slot is claimed. So once a key has had one early signal lapse and its next run does not come (a retired workflow, or one that runs monthly), every pass from then on:
- lists the whole
wait-pending/prefix, - calls
listKeyRecords()over the whole store, which is one read per key record of every workflow, - reads
highestKeyRecordandhighestPendingRecordfor each slot.
Nothing is wrong with the result. On a control-plane store backed by S3 or GCS it is a steady stream of requests that never stops and grows with the number of keyed workflows. findTakerOf in src/domain/workflows/early_signal.ts has the same shape (it lists every key record of every workflow), but only runs on a rare arrival path.
Suggested fix
- Skip a slot whose highest record is already a lapse with nothing below it before
listKeyRecords()is called. - Build the set of claims that read a slot lazily, only when a slot that is not live turns up.
- Or run this part of the sweep on a longer cadence than the continuation pass.
- Consider a per-key listing for
findTakerOfinstead oflistKeyRecords().
Assert on reads done, not on time: a test with only lapsed slots should make no call to listKeyRecords.
Shipped
Click a lifecycle step above to view its details.
Sign in to post a ripple.