Skip to main content

KEEP THE TOKEN SECRETS KEY OUTSIDE THE DATASTORE

This guide shows you how to move the key that encrypts the _token-secrets control-plane vault out of the datastore on an existing swamp serve deployment. By default the key is stored beside the secrets it encrypts, so anyone with read access to the datastore can decrypt every server token.

Warning

There is no way back to a key stored in the datastore. Once serve has moved the control plane to an external key, every instance and every host that runs swamp access token commands locally needs that key.

Prerequisites

  • A running swamp serve deployment
  • A vault whose storage is outside the datastore, readable with the same value on every serve instance and on every host that runs swamp access token mint, rotate or reveal, or swamp worker token create, locally. A local_encryption vault keeps its key in the always-local .swamp/secrets/, so it fits a single host; a multi-instance deployment needs a shared external vault
  • openssl, or another CSPRNG

1. Generate the key and store it in the vault

Generate a 32-byte key and store it without printing it:

$ openssl rand -base64 32 | swamp vault put prod-secrets swamp-token-secrets-key --yes
[INF] vault·put: Stored secret "swamp-token-secrets-key" in vault "prod-secrets"

The key may be hex- or base64-encoded. Swamp never generates it, and rejects a key with the same value in every byte.

2. Add the block to the serve config

Add a token-secrets: block to each instance's serve config file (.swamp/serve.yaml by default), and to .swamp/serve.yaml on every host that runs the token commands locally:

token-secrets:
  vault: prod-secrets
  key: swamp-token-secrets-key

3. Check the config

$ swamp serve check-config
Auth mode: token
  No usernames to resolve in this mode.

Token secrets key: vault prod-secrets, key swamp-token-secrets-key
  ✓ resolves to a usable 32-byte key

Result: PASSED

check-config never contacts the datastore. If the vault's config arrives through the datastore, sync it first.

4. Restart every instance together

Stop every swamp serve instance, then start them all with the block in place. An instance still running on the old key while another migrates can write a secret that neither key opens.

On its first start with the block, serve re-encrypts every token secret with the external key, then replaces the key in the datastore with a marker that names the vault and key. Entries that neither key decrypts are left as they are and logged by name.

5. Rotate old tokens and delete old copies

The move cannot reach copies of the token secrets made before it. Each still holds the old key:

  • datastore backups
  • noncurrent object versions on a versioned bucket
  • a root-level _control/token-secrets/ left by a namespace migration

Rotate every token minted before the move:

$ swamp access token rotate <name>

Then delete those copies. Serve logs a warning after migrating, and an error on every namespaced start while a key stored in the datastore remains at the root.

← PreviousDeploy swamp serve on KubernetesNext →Set Up Extension Trust