Relationships
#4889 serve hangs forever if its shutdown throws before the listener closes
Opened by zack · 11/18/2026· Shipped 11/18/2026
Description
shutdown() in src/cli/commands/serve.ts has no fallback. If any step throws before listenerAc is aborted, the serve action keeps waiting on server?.finished, which never resolves. Every trigger shares the memoized shutdownRun, so another SIGTERM or stdin close cannot retry. Serve stays up indefinitely, keeps its port, and /ready answers 503 shutting_down.
This is how swamp-club#4885 hung. There a stopping line written through the console guard to a broken stderr threw BrokenPipe. swamp-club#4884 and swamp-club#4885 removed every throwing write known on that path. A future throwing write or unstaged step would bring the hang back.
Why the obvious fallback is not enough
An onFailed hook that aborts the listener was tried in swamp-club#4885 and dropped in review.
- Returning normally: the CLI runs its success teardown over runs that may still be writing. That covers retention compaction, follow compaction, and removing the v3 writer markers.
- Failing the command instead:
closeV3DatastoresAfterFailurestill compacts and releases the lease. Meanwhile retention GC, v3 maintenance, replication and runs, which shutdown never stopped, may still be running. That breaks serve's rule that their work never overlaps the closing compaction (serve.ts, the comments beside the retention and maintenance stops).
Possible directions
- Crash-style exit: keep the markers and exit at once, as a SIGKILLed serve would. Recovery already handles that state.
- Best-effort stop: run a bounded stop of retention GC, v3 maintenance and replication before failing the command.
- Skip compaction on failure: have the failure teardown skip compaction when the writer markers are kept.
- Run more of shutdown under ShutdownStages: it never throws, which shrinks the code that can throw.
Environment
swamp main at 8408c7bf, found while working on swamp-club#4885.
Shipped
Click a lifecycle step above to view its details.
Sign in to post a ripple.