Relationships
#2473 serve: remote token mint impossible with multiple vaults -- server demands --vault, CLI has no --vault option
Opened by randybias · 9/24/2026
Problem
With more than one vault configured, a token cannot be minted through --server at all.
$ swamp access token mint probe --principal user:probe --duration 15m --server wss://... --json
Server reported access_token_mint_failed: Multiple vaults are configured (mircloud-colo, _token-secrets). Pass --vault <name> to choose one.
$ swamp access token mint probe ... --vault _token-secrets --server wss://...
Unknown option "--vault".The server asks for --vault, and the CLI has no such option (20260923.231117.0; swamp help access token mint lists none). Separately, access_token_mint.ts rejects a vault together with --server (tokenVaultRejectedMessage(... {remote: true})).
Why it matters
A mint done in-process through --server is exactly what --grant-reload auto picks up live. Blocking it forces operators to mint from a separate CLI process on the server host (kubectl exec, for us). The running server never sees that mint, so every new operator or automation token costs a server restart. For a single-replica deployment that restart is an outage. We have done it several times in one day.
Ask
- Accept
--vaultonaccess token mintover--server, and pass it in theaccess.token.mintpayload. Or let serve config name the vault used for token secrets. - Make the error message name an option that exists.
Closed
No activity in this phase yet.
skunk-ape commented 9/24/2026, 2:14:12 PM
Thanks @randybias for this report, and especially for spelling out the operational cost: a restart per token, which is an outage on a single replica. That shaped how we tested the fix.
This is the same defect as #2466, which is now fixed in swamp-club/swamp#2606 and released as 20260924.140802.0-sha.daa5bb2c. Closing this as a duplicate.
On your two asks:
- Accept
--vaultover--server, or a serve setting for the token vault. We did not add either. The serve now always stores remote-minted token secrets in the_token-secretscontrol-plane vault, as a local mint does, so no remote caller has to pick a vault. Keeping the vault a server-side constant also means a remote client cannot redirect where token secrets are stored. - Make the error name an option that exists. Remote mint can no longer produce that error.
On restarts: a token minted with --server is live on the running serve at once, with no restart. The restarts came from minting in a separate process (kubectl exec). With a remote datastore, a running serve does not pull the out-of-process token definition, so that token failed until the next boot. Minting through --server avoids that.
Two things to know:
- The mint output does not include the credential, and
swamp access token revealhas no--server. Runrevealwhere the serve repo or its datastore is reachable. That is a read, so it needs no restart. - With more than one replica, a mint reaches only the replica that handled it. The other replicas still need a restart to see it. Your single-replica setup is fully covered.
The new token still needs a grant before it can do anything. --grant-reload auto or swamp access reload picks that up live, as before.
Sign in to post a ripple.