#3405 serve keeps a worker whose enroll finished after the worker timed out, then refuses the real worker as "already connected" and dispatches into the dead session

Description

Right after a serve restart, enrollment was slow (30+ s while the whole fleet reconnected). A worker's worker.enroll RPC timed out on the client after 30 s, and the worker closed that socket and tried again. Serve still finished each enrollment a few seconds later and kept the last one as a live worker, even though its socket was already gone.

After that:

• every new connection from the real worker is rejected with Worker '<name>' is already connected, and the worker gives up after 3 attempts and loops (systemd restarts it);
• serve shows the dead session as busy in swamp worker list, with lastSeenAt still updating;
• serve kept dispatching to it. A release's build step was re-dispatched to the dead session every ~7 s with a new dispatch ID: 71 dispatches over 25+ minutes, none started anywhere. The step stayed running, and swamp worker verify probes routed there timed out.
• Two healthy idle workers with the same labels were never chosen.

It didn't clear on its own (--ws-idle-timeout is the 30 s default) and swamp worker prune doesn't touch it. The only way out we found is restarting serve again, which ends any running workflow.

Steps to reproduce

1. Serve with ~15 workers connected (here 11 GKE fleet workers and 3 GCE build workers).
2. systemctl restart swamp-serve. Every worker reconnects and enrolls at once.
3. A worker whose enroll takes longer than its 30 s RPC timeout:
   • worker: Connecting… (attempt 1) at 20:56:47, then Disconnected: RPC call 'worker.enroll' timed out after 30000ms at 20:57:17;
   • serve: Worker "swamp-build-worker-v3-a537ddf2" enrolled ("first-connect") at 20:57:22.
   • The same happened for attempts 2 and 3: the worker timed out at 20:57:49 and 20:58:21; serve recorded enrolled ("reconnect") at 20:58:04 and 20:58:23.
4. From 20:58:35 on, every attempt by the worker: Disconnected: Worker 'swamp-build-worker-v3-a537ddf2' is already connected.
5. Run a workflow whose step matches that worker's labels. Serve dispatches to the dead session in a loop: Dispatching "@swamp/ci-artifact/builder"."build" (step "build") to worker "swamp-build-worker-v3-a537ddf2", about every 7 s.

Expected

• If the enroll reply can't be delivered because the client socket is closed, serve drops the registration rather than keeping it.
• A new connection that proves the same identity (same enrollment token and machine id) replaces the old session instead of being refused, or serve checks the old socket first.
• A dispatch that is never acknowledged stops being retried on the same worker. It goes to another matching worker, or fails the step with a clear error.

Environment

• serve 20261013.004531.0-sha.1373120f on a GCE VM (local filesystem datastore, behind a Google HTTPS load balancer)
• build workers 20261013.004531.0-sha.1373120f; GKE fleet workers v5
• All workers connect to wss://ops.swamp-club.com.

Related: #3399, filed today about the per-IP WebSocket burst limit, which adds to reconnect storms on shared-IP networks.