Relationships
#4720 worker token rotate counts a machine as retired after it took an earlier rotation in-band and disconnected
Opened by zack · 11/13/2026· Shipped 11/13/2026
swamp worker token rotate (through --server) over-counts retiredBindings for a machine that took an earlier rotation over its own connection and then disconnected.
Found while documenting rotate for swamp-club#4715, on swamp 20261113.151221.0-sha.15077209.
Repro (local swamp serve, fleet token, one worker started with --token-file):
- Worker enrolls on mint A. Rotate through serve: the worker is rotated and persisted to mint B, but the token record's binding for that machine still says mint A (swamp worker token list --json).
- Rotate again through serve while the worker is connected: rotated to mint C, persisted. The binding still says mint B.
- Stop the worker. Rotate a third time. The report says the machine "still hold[s] a token an earlier rotation retired — they need the new token to enroll", counted in retiredBindings.
- Restart the worker from its token file (mint C, the current predecessor). It enrolls normally, so it was never retired.
Cause, from the source: WorkerGateway.rotateToken classifies unattended bindings by record.bindings[].mint, and a binding's mint moves only on redeem (the code comments say so). An in-band rotation acknowledged by the worker restamps only the in-memory pool entry, so once that worker disconnects its binding carries a stale mint and isRetired treats it as retired.
Effect: the report tells the user a machine needs the new token to enroll when it does not, and inflates retiredBindings in --json. Worth checking whether anything else reads the binding mint for a machine in that state (revalidation on an HA peer, enrollmentTokenBindingVerdict).
Expected: an acknowledged in-band rotation also records the new mint on that machine's binding, or the classification uses the latest mint the machine acknowledged.
Shipped
Click a lifecycle step above to view its details.
Sign in to post a ripple.