WORKER COMMANDS

CLI reference for managing remote execution workers and the Swamp server. This page covers token lifecycle commands, worker enrollment, pool inspection, queue visibility, and the swamp serve entry point.

For enrollment token lifecycle details (machine binding, expiry, state transitions), see Enrollment Tokens. For step-level placement fields that route work to specific workers, see Workflow Placement.

Global flags

All commands in this section accept the standard global options. These are documented once here rather than repeated per command.

swamp worker token create

Mint a named worker enrollment token. The plaintext is shown exactly once in <name>.<secret> format.

Locally, the token secret is always stored in the _token-secrets control-plane vault. With --server, serve stores the plaintext in a vault on the server: the one named by --vault, or the control-plane vault _token-secrets when --vault is omitted. If the control-plane vault cannot initialize, the command fails; see Control-plane vault failures.

By default the key that encrypts _token-secrets is stored in the datastore beside the secrets. Run locally, the command reads the token-secrets: block from .swamp/serve.yaml, and refuses to run without it once serve has moved the control plane to an external key. See Token secrets encryption key.

swamp worker token create <name> --duration <duration> [--vault <vault>] [--max-enrollments <n>]

Flags

Example

$ swamp worker token create ci-runner --duration 24h

Token: ci-runner
Expires: 2026-06-16T10:34:17.833Z
Vault: _token-secrets (key worker-token-ci-runner)

  ci-runner.9ce100bf00607d0edc3f231d6f7d85d028ba820b238fda113469cf0fdf7bfa97

This token is shown once and will not be displayed again — store it now.

The token value (ci-runner.9ce100bf...) is the credential passed to swamp worker connect --token. The name prefix before the dot identifies the token; the suffix after the dot is the secret. See Enrollment Tokens for the full token lifecycle.

Fleet token example

$ swamp worker token create gpu-fleet --duration 7d --max-enrollments 10

Token: gpu-fleet
Expires: 2026-07-11T00:00:00.000Z
Max enrollments: 10
Vault: _token-secrets (key worker-token-gpu-fleet)

  gpu-fleet.a4e29c1f00817b3eac5f642d7e8c96b139ca931b349fea224580de1fef8cbaa8

This token is shown once and will not be displayed again — store it now.

swamp worker token list

List worker enrollment tokens showing recorded vs effective (display) state, expiry, and enrollments.

swamp worker token list

Flags

Table output

$ swamp worker token list

NAME        STATE     EXPIRES                   ENROLLMENTS
ci-runner   enrolled  2026-06-16T10:34:17.833Z  machine-42
gpu-fleet   enrolled  2026-07-11T00:00:00.000Z  3 / 10

Single-enrollment tokens show the bound machine ID (e.g. machine-42). Fleet tokens show used / allowance (e.g. 3 / 10, 8 / unlimited).

JSON output

$ swamp worker token list --json

[
  {
    "name": "ci-runner",
    "state": "enrolled",
    "effectiveState": "enrolled",
    "createdAt": "2026-06-15T10:34:17.833Z",
    "expiresAt": "2026-06-16T10:34:17.833Z",
    "maxEnrollments": 1,
    "bindingCount": 1,
    "bindings": [
      { "machineId": "machine-42", "enrolledAt": "2026-06-15T10:34:30.000Z" }
    ],
    "vaultName": "_token-secrets",
    "secretKey": "worker-token-ci-runner"
  }
]

JSON fields

swamp worker token revoke

Invalidate a worker enrollment token before it expires, and cut off every worker bound to it. With --server, the orchestrator disconnects them at once; a local revoke reaches a running orchestrator at its next token check. See Revocation.

swamp worker token revoke <name>

Flags

Example

$ swamp worker token revoke ci-runner

✓ Token ci-runner revoked.
Revoked at: 2026-06-15T10:34:39.736Z
A running orchestrator disconnects workers on this token at its next token check (about every 30 seconds).

Through the server, the output names the workers it disconnected:

$ swamp worker token revoke ci-runner --server wss://swamp.example.com

✓ Token ci-runner revoked.
Revoked at: 2026-06-15T10:34:39.736Z
Disconnected workers: ci-runner-3f2a, ci-runner-9b1c

When no connected worker was enrolled on the token, the last line reads No connected workers were enrolled on this token.

With --json, a revoke through the server adds disconnectedWorkers; a local revoke omits it, because it cannot see the live pool:

{
  "name": "ci-runner",
  "state": "revoked",
  "revokedAt": "2026-06-15T10:34:39.736Z",
  "alreadyRevoked": false,
  "disconnectedWorkers": ["ci-runner-3f2a", "ci-runner-9b1c"]
}

After revocation, swamp worker token list reflects the updated state:

NAME       STATE    EXPIRES                   ENROLLMENTS
ci-runner  revoked  2026-06-16T10:34:17.833Z  -

swamp worker connect

Connect this machine to an orchestrator as a remote execution worker.

swamp worker connect [url] --token <token> [--label k=v ...] [--cache-dir <dir>] [--data-plane-url <url>] [--concurrency <n>] [--no-reconnect]

Flags

Cache directory

Set --cache-dir to a stable directory so the worker can re-enroll after a restart. Without it, the worker receives a fresh temp directory (and therefore a new machine identity) on every process start. A stable cache directory preserves the machine-id file that the enrollment token binds to, allowing the worker to resume its identity across restarts.

Environment variables

Explicit flags take precedence when both a flag and its environment variable are set.

Without a swamp-club credential, the worker passes the account check on a pass from the serve that enrolls it. See Remote workers.

Workers older than CLI release v20260922.185723.0-sha.33229b53 apply the CA certificate to the control connection only, so dispatches fail with fetch failed. Set DENO_CERT to the same file for those workers.

Examples

$ swamp worker connect wss://orchestrator.example.com \
    --token ci-runner.9ce100bf00607d0edc3f231d6f7d85d028ba820b238fda113469cf0fdf7bfa97 \
    --label env=ci --label arch=arm64 \
    --cache-dir /var/lib/swamp-worker

# Container/cloud-init: connect using only environment variables
SWAMP_ORCHESTRATOR_URL=wss://orch:9090 \
SWAMP_WORKER_TOKEN=ci-pool.a3f9... \
SWAMP_WORKER_LABELS=tier=ci,region=us-east \
SWAMP_WORKER_CACHE_DIR=/var/lib/swamp-worker \
swamp worker connect

# Ephemeral worker: one dispatch, then exit
swamp worker connect wss://orch:9090 --token <token> --max-dispatches 1

# Exit after 10 idle minutes
swamp worker connect wss://orch:9090 --token <token> --idle-timeout 10m

# Big node: 8 concurrent dispatch slots
swamp worker connect wss://orch:9090 --token <token> --concurrency 8

# Auto-detect from CPU count
swamp worker connect wss://orch:9090 --token <token> --concurrency auto

# Connect to a token-authenticated orchestrator
swamp worker connect wss://orch:9090 \
    --server-token admin.secret \
    --token worker-pool.enrollment-secret

# Read tokens from files instead of passing them inline
swamp worker connect wss://orch:9090 \
    --token-file /run/secrets/worker-token \
    --server-token-file /run/secrets/server-token

Server token vs enrollment token

--token is the enrollment token — it identifies the worker to the orchestrator and binds it to a machine identity. --server-token is the server access token — it authenticates the WebSocket connection itself. A server token is required when the orchestrator runs with --auth-mode token. Without it, the connection receives HTTP 401.

Use swamp auth server-login to store a server token for reuse, or set the SWAMP_SERVER_TOKEN environment variable.

Both --token and --server-token accept a file-based alternative: --token-file and --server-token-file read the token from a file path instead of taking it inline. This avoids exposing secrets in process listings or shell history. Each flag is mutually exclusive with its inline counterpart — passing both is an error.

swamp worker list

List workers in the pool. By default, only connected and draining workers are shown. Disconnected workers are hidden unless --all is passed.

swamp worker list [--all] [--server <url>] [--token <token>]

Flags

The STATUS column reports connected, disconnected, or draining. A draining worker is finishing in-flight work before disconnecting and is excluded from new dispatch scheduling. Without --all, only connected and draining workers appear.

The SLOTS column shows the worker's current dispatch load as active/capacity (e.g. 0/4, 3/8). Workers with --concurrency 1 (the default) show 0/1 or 1/1.

Table output

$ swamp worker list

NAME            STATUS     LABELS          PLATFORM      SLOTS  LAST SEEN
ci-runner-a1b2  connected  env=ci          linux/amd64   0/1    2026-07-04T10:34:30.000Z
gpu-fleet-c3d4  connected  gpu=true        linux/arm64   3/8    2026-07-04T10:34:31.000Z

Table output with --all

$ swamp worker list --all

NAME            STATUS        LABELS          PLATFORM      SLOTS  LAST SEEN
ci-runner-a1b2  connected     env=ci          linux/amd64   0/1    2026-07-04T10:34:30.000Z
gpu-fleet-c3d4  connected     gpu=true        linux/arm64   3/8    2026-07-04T10:34:31.000Z
build-node-e5f6 disconnected  tier=build      linux/amd64   0/4    2026-07-03T22:15:00.000Z

JSON output

$ swamp worker list --json

{
  "workers": [
    {
      "name": "ci-runner-a1b2",
      "tokenName": "ci-fleet",
      "status": "connected",
      "labels": { "env": "ci" },
      "platform": "linux",
      "arch": "amd64",
      "instanceUuid": "3f9c2e1a-7b4d-4c8e-9a10-5d2f6b8e1c47",
      "enrolledAt": "2026-07-01T09:12:00.000Z",
      "lastSeenAt": "2026-07-04T10:34:30.000Z",
      "capacity": 1,
      "activeDispatchIds": []
    }
  ],
  "count": 1,
  "filteredDisconnectedCount": 1
}

filteredDisconnectedCount is the number of disconnected workers omitted from the workers array. With --all, this field is 0 and all workers are included.

JSON fields (envelope)

JSON fields (per worker)

Empty pool

When the pool is empty:

$ swamp worker list

No workers found.
Workers appear here after enrolling with: swamp worker token create <name> --duration 24h

swamp worker verify

Run a fleet probe against connected workers to confirm dispatch, capability RPC, and data-plane connectivity.

swamp worker verify [name] [--label k=v] [--server <url>]

Arguments

Flags

Exits non-zero when any worker fails verification.

Table output

$ swamp worker verify

WORKER         STATUS  PLATFORM  ARCH    DETAILS
ci-runner-a1b2 pass    linux     amd64   -
gpu-fleet-c3d4 fail    linux     arm64   dataPlaneOk: false

JSON output

$ swamp worker verify --json

{
  "workers": [
    {
      "name": "ci-runner-a1b2",
      "status": "pass",
      "probeMarkerOk": true,
      "queryOk": true,
      "dataPlaneOk": true,
      "failures": []
    },
    {
      "name": "gpu-fleet-c3d4",
      "status": "fail",
      "probeMarkerOk": true,
      "queryOk": true,
      "dataPlaneOk": false,
      "failures": ["data plane write+read round-trip failed"]
    }
  ],
  "total": 2,
  "passed": 1,
  "failed": 1
}

JSON fields

swamp worker queue

List steps currently queued for dispatch, waiting for a matching worker.

swamp worker queue [--server <url>] [--repo-dir <dir>]

When steps are waiting:

$ swamp worker queue

REQUIREMENT          STEP              MODEL          QUEUED AT                   AGE
labels gpu=true      train-model       ml/trainer     2026-07-04T10:32:15.000Z    3m
target build-node    compile           builder/go     2026-07-04T10:34:02.000Z    1m

When the queue is empty:

$ swamp worker queue
No steps are currently queued.

JSON output

$ swamp worker queue --json

[
  {
    "queueId": "a1b2c3d4-...",
    "requirement": "labels gpu=true",
    "stepName": "train-model",
    "modelType": "ml/trainer",
    "methodName": "run",
    "workflowName": "training-pipeline",
    "queuedAt": "2026-07-04T10:32:15.000Z",
    "ageMs": 180000
  }
]

JSON fields

Steps appear here when they are waiting for a matching worker and disappear once dispatched, timed out, or cancelled. Use this to diagnose why a workflow is stalled and to inform autoscaling decisions.

The same data is queryable via swamp data query:

swamp data query 'modelType == "swamp/pending-dispatch" && attributes.state == "waiting"'

swamp worker prune

Remove stale workers from the pool. A worker is stale when it has been disconnected longer than the grace period. Useful in ephemeral fleet deployments where workers come and go and disconnected entries accumulate.

swamp worker prune [--grace-period <duration>] [--dry-run] [--force] [--server <url>] [--token <token>]

Flags

Dry-run output

$ swamp worker prune --dry-run

DRY RUN — no workers will be removed

NAME              STATUS        LAST SEEN                   AGE
build-node-e5f6   disconnected  2026-07-03T22:15:00.000Z    2d
old-runner-a1b2   disconnected  2026-07-01T08:00:00.000Z    4d

2 workers would be pruned (grace period: 24h)

Example

$ swamp worker prune --grace-period 7d --force

Pruned 1 stale worker (grace period: 7d)
  old-runner-a1b2  disconnected since 2026-07-01T08:00:00.000Z

JSON output

$ swamp worker prune --dry-run --json

{
  "dryRun": true,
  "gracePeriod": "24h",
  "pruned": [
    {
      "name": "build-node-e5f6",
      "status": "disconnected",
      "lastSeen": "2026-07-03T22:15:00.000Z"
    }
  ],
  "count": 1
}

Only disconnected workers are candidates for pruning. Connected and draining workers are never removed regardless of the grace period.

swamp worker daemon enable

Generate and install a systemd unit (Linux) or launchd plist (macOS) that runs swamp worker connect as a system service. Every worker connect setting is written into the service, including SWAMP_SERVE_EXTRA_HEADERS from the shell that runs enable.

swamp worker daemon enable [url] --token <token> [--user] [--label k=v ...] [--cache-dir <dir>] [--max-dispatches <n>] [--idle-timeout <dur>] [--concurrency <n|auto>] [--ca-cert <path>]

Flags

Paths

--token-file, --server-token-file, --cache-dir and --ca-cert are made absolute against the directory enable runs in. The service runs from the cache directory or /, so the unit records the absolute path.

--ca-cert must name an existing, readable file. A system service runs as another user, so that user must also be able to read it.

Enable-time checks

enable refuses, before writing anything, every value swamp worker connect would refuse when the service starts:

Labels

The service receives labels as one comma-separated SWAMP_WORKER_LABELS value. A --label whose value contains a comma, or spaces the service trims, still installs, but enable warns that the service reads it as different labels. For example, --label a=x,y=z reaches the service as two labels, a=x and y=z.

A --label the service cannot read back at all, such as a=b,c, is refused, naming the offending --label.

Extra headers

SWAMP_SERVE_EXTRA_HEADERS set in the shell that runs enable is written into the service, as worker connect would read it in that shell. enable prints the header names, never their values:

✓ Worker daemon enabled as system service — starts automatically at boot
  Extra headers from SWAMP_SERVE_EXTRA_HEADERS: Tunnel-Token

With --json, the names are in extraHeaderNames.

sudo drops the variable by default. Run enable as sudo -E or sudo --preserve-env=SWAMP_SERVE_EXTRA_HEADERS, or the service is written without the headers.

Service settings

The generated unit file configures the following behavior:

Token and other secrets are placed in environment directives inside the unit file, which is installed with mode 0600. They do not appear in ExecStart (systemd) or ProgramArguments (launchd).

--cache-dir should point to a stable path (e.g. /var/lib/swamp-worker) for daemon mode so the machine identity survives service restarts.

The unit also sets SWAMP_CONFIG_DIR to the enabling user's config directory, so the service reads that user's swamp-club credential. Run swamp auth login as that user before enable, which needs a credential of its own. A worker daemon enabled before the account requirement has no SWAMP_CONFIG_DIR. Run as a system service that finds no credential, it passes through its serve as a remote worker; run enable again to regenerate the unit and use the user's credential.

Example

swamp worker daemon enable wss://orch:9090 \
  --token pool.secret \
  --label tier=ci --label region=us-east \
  --cache-dir /var/lib/swamp-worker \
  --idle-timeout 30m

# Token-authenticated orchestrator
swamp worker daemon enable wss://orch:9090 \
  --token pool.secret \
  --server-token admin.secret \
  --label tier=ci

# Orchestrator with a private-CA certificate, one slot per CPU
swamp worker daemon enable wss://orch.internal:9090 \
  --token-file /etc/swamp/worker-token \
  --ca-cert /etc/swamp/ca.pem \
  --concurrency auto

# Reverse proxy that requires a tunnel token
export SWAMP_SERVE_EXTRA_HEADERS="Tunnel-Token: abc123"
sudo --preserve-env=SWAMP_SERVE_EXTRA_HEADERS \
  swamp worker daemon enable wss://orch.internal:9090 --token pool.secret

swamp worker daemon disable

Stop the service and remove the unit file.

swamp worker daemon disable

swamp worker daemon status

Show whether the daemon is installed, running, its PID, and log location.

swamp worker daemon status [--json]

Table output

$ swamp worker daemon status

Daemon:   installed
State:    running
PID:      48201
Log:      /var/log/swamp-worker.log

JSON output

$ swamp worker daemon status --json

{
  "installed": true,
  "running": true,
  "pid": 48201,
  "logPath": "/var/log/swamp-worker.log"
}

swamp serve

Start a WebSocket API server for workflow and behavior execution. Workers connect to this server.

swamp serve [--port <port>] [--host <host>] [--cert-file <path>] [--key-file <path>] [--no-schedule] [--webhook <spec>] [--repo-dir <dir>]

Flags

When both --cert-file and --key-file are provided, the server listens over wss:///https:// instead of ws:///http://. See TLS and Proxies for details.

Service deployments

Swamp loads all extensions through an embedded runtime under ~/.swamp. When running under a service manager (e.g. systemd), ensure HOME is set in the unit environment so the runtime can locate its extension store.

Example

$ swamp serve --port 8080 --host 0.0.0.0 --webhook /deploy:ci-deploy:s3cr3t

Exit-code contract

swamp worker connect uses its exit code to distinguish intentional shutdown from failure.

See Failure Semantics — Graceful drain for the full drain sequence.

Related

• Enrollment Tokens — token lifecycle, machine binding, expiry
• Workflow Placement — step placement fields and scheduling
• Failure Semantics — reconnection, in-flight dispatch, cancellation
• Remote Execution — overview and core concepts
• TLS and Proxies — trust stores and server-side TLS for swamp serve
• Serve Flags — all swamp serve flags including authentication and access control