Skip to main content
← Back to list
01Issue
FeatureShippedSwamp ClubPublic
AssigneesNone

Relationships

↔ sibling #4577

#4900 Docs: team serve rolling restart runbook, leader takeover time, and how clients treat an inFlight refusal (#4811, #4824, #4830, #4839, #4853)

Opened by stack72 · 11/19/2026· Shipped 11/19/2026

Three shipped swamp serve changes from the #4439 reload and restart follow-ups have no docs issue, and the manual under content/manual/ (swamp-club repo) does not cover them. Docs only. The open docs issues #4577, #4794 and #4829 touch the same restart pages, so these fit in the same docs PR.

1. Restarting a team serve one host at a time (#4811)

The manual never mentions a rolling restart. #4811 asked for an operator runbook:

  • Restart followers first, one at a time. A follower owns no runs, so nothing is parked; its connections move to the other hosts behind the load balancer.
  • Restart the leader last. It parks its runs at a step boundary (#4737) and gives up the writer lease; a follower is promoted and continues the parked runs under the same run ids.
  • What to watch between hosts: wait for the restarted host's /ready before moving on, and use /ready, not /health, as the load balancer check (see #4577).

Pages: how-to/swamp-serve/run-multi-instance-deployment.md (a rolling restart section), linked from how-to/swamp-serve.md. #4325 (team mode manual pages) covers the same pages more broadly.

2. How long a leader takeover takes (#4824)

The writer lease now has a 15 s TTL, so after a leader dies a follower takes over in about 25 s, down from about 40 s. A planned leader restart hands the lease over without waiting for it to lapse. The multi-instance page explains takeover but gives no time; say how long writes and run starts are refused while a dead leader's lease lapses, and that a member's CLI rides it out with its read retries (about 15 s) and run-start retries (30 s).

Page: how-to/swamp-serve/run-multi-instance-deployment.md.

3. What a client does with an inFlight refusal (#4830, #4839, #4853)

The multi-instance page explains a follower's serve_not_leader with inFlight: true (the request may already have run). Add:

  • swamp ... --server exits 1 for an inFlight refusal, not 75: exit 75 means safe to retry, and this request may have run. A refusal without inFlight still exits 75.
  • An HTTP action (build-an-integration.md) answers an inFlight refusal as outcome_unknown (503), never as a retryable serve_not_leader. @swamp-club/swamp-lib never resends it; read the run's status before retrying.
  • A leader marks a not-leader refusal raised after a request started as inFlight too, with a message that it lost the lead while handling the request.
  • A follower that names no leader (the data plane) still says the request may already have run for an inFlight refusal, never try again.

Pages: how-to/swamp-serve/run-multi-instance-deployment.md, how-to/swamp-serve/build-an-integration.md (the error table), and wherever the manual lists exit 75.

02Bog Flow
✓OPEN✓TRIAGED✓IN PROGRESS✓SHIPPEDLINKED+ 3 MOREPR_LINKED+ 1 MORESESSION_SUMMARIZED

Shipped

11/19/2026, 6:06:02 PM

Click a lifecycle step above to view its details.

03Sludge Pulse
stack72 linked sibling of #457711/19/2026, 5:05:35 PM
Editable. Press Enter to edit.

system commented 11/19/2026, 4:57:59 PM

Classified automatically when this issue was filed.

  • Source: Swamp Club

If you feel this classification is incorrect, add a ripple to tell us so.

Sign in to post a ripple.