Skip to main content
← Back to list
01Issue
BugShippedSwamp CLIPublic
Assigneeszack

Relationships

#4558 serve: a vault outage during swamp serve reload removes every working webhook route file

Opened by zack · 11/9/2026· Shipped 11/9/2026

Problem

Webhook route files (swamp-club#4547) are parsed one file at a time on every startup and swamp serve reload, so a bad file is refused alone. Parsing a route file includes resolving its @vault= secret. If the vault cannot be read for a moment during a reload (backend timeout, transient error), every route file fails to parse and is refused. The reload then calls WebhookService.updateEndpoints with only the operator routes, so every previously working file route is removed and its POSTs stop matching until the next successful reload. The reload still reports success; the refusals appear only as reload errors.

Operator routes behave differently: serve.yaml webhooks parse all-or-nothing, so a vault failure fails the webhook update and the live routes stay as they were.

Steps to reproduce

  1. Serve with one or more working route files in webhook-routes/ (and no serve.yaml webhooks, or flag mode).
  2. Make the vault named by their secrets unreadable (stop the backend, or make reads time out).
  3. Run swamp serve reload.
  4. Every file route is reported refused and removed; POSTs to those routes return 404 until the vault recovers and serve reloads again.

Expected

A failure that is about the environment rather than the file (vault unreachable, a transient scheme-resolution failure) keeps the route's previous endpoint, while a failure that is about the file itself (validation, a missing key in a reachable vault) still refuses it. At minimum the behaviour is documented.

Notes

  • Code: buildWebhookRouteTable and parseRouteFiles in src/serve/webhook_reload.ts and src/serve/webhook_route_files.ts; parseWebhookConfigs in src/cli/commands/serve.ts.
  • Keeping the previous endpoint needs WebhookService to expose the current endpoint for a route (with its resolved secret) to the table builder.
  • Raised by the code and adversarial reviews during swamp-club#4547 verification.
02Bog Flow
✓OPEN✓TRIAGED✓IN PROGRESS✓SHIPPED+ 1 MOREASSIGNED+ 5 MOREREVIEW+ 10 MOREPR_LINKED+ 2 MORESESSION_SUMMARIZED

Shipped

11/9/2026, 7:51:40 PM

Click a lifecycle step above to view its details.

03Sludge Pulse
zack assigned zack11/9/2026, 7:02:47 PM

Sign in to post a ripple.