RUN A FLEET WITH DOCKER COMPOSE
This guide shows you how to deploy a pool of Swamp workers with Docker Compose, using a single fleet token to enroll multiple containers.
Prerequisites
swamp auth logincompleted orSWAMP_API_KEYset withserve:*scope (for the orchestrator)- An orchestrator on Swamp
v20261008.183339.0-sha.231eda34or later, started while swamp-club.com is reachable. It vouches for the workers it enrolls, so they need no swamp-club.com key — see Remote workers - A running orchestrator with TLS and token authentication. See
Run a Worker in Docker for the
orchestrator setup steps (TLS certificate,
swamp serve, and theworker-tokenserver token) - The
swamp-workerimage, built from the Dockerfile in Run a Worker in Docker
Create a fleet token
Create an enrollment token that allows multiple machines to enroll:
swamp worker token create compose-fleet --duration 7d --max-enrollments 20Save the token value. Each container that connects with this token receives an
auto-generated name (compose-fleet-<suffix>) and a fleet=compose-fleet
label.
The token expires after seven days. To keep the fleet past then, follow
Rotate a Worker Enrollment Token.
The Compose file passes the token in SWAMP_WORKER_TOKEN, so a rotation reaches
running containers in memory only: put the new token in the Compose file, or
restarted containers present the old one.
Write the Compose file
services:
worker:
image: swamp-worker
environment:
SWAMP_ORCHESTRATOR_URL: wss://host.docker.internal:9090
SWAMP_SERVER_TOKEN: worker-token.<secret>
SWAMP_WORKER_TOKEN: compose-fleet.<secret>
SWAMP_WORKER_CONCURRENCY: "4"
SWAMP_CA_CERT: /certs/cert.pem
volumes:
- ./cert.pem:/certs/cert.pem:ro
command: ["worker", "connect"]
stop_grace_period: 5m
restart: on-failureOn Linux with Docker Engine rather than Docker Desktop, host.docker.internal
does not resolve inside a container by default. Add it to the worker service:
extra_hosts:
- "host.docker.internal:host-gateway"For an orchestrator older than v20261008.183339.0-sha.231eda34, or an image
based on Alpine, give the workers their own key: add
SWAMP_API_KEY: ${SWAMP_API_KEY} and
SWAMP_SIGNIN_TOKEN: ${SWAMP_SIGNIN_TOKEN} to environment, set from a
collective token in the shell or an .env file beside the Compose file.
stop_grace_period: 5m gives in-flight steps five minutes to complete when
Docker sends SIGTERM during docker compose down or scaling events. If a step
takes longer than the grace period, Docker force-kills the container and the
step is lost.
restart: on-failure restarts a worker that crashes but does not restart one
that exits cleanly (exit code 0). A clean exit happens when --max-dispatches
or --idle-timeout triggers a drain.
Workers can start before the orchestrator, and ride out its restarts: a worker
keeps retrying for up to 5 minutes while the orchestrator refuses connections or
answers 502–504. See
While the orchestrator restarts.
SWAMP_WORKER_CONCURRENCY sets the number of concurrent dispatch slots per
container. Adjust based on the workload — CPU-bound steps benefit from fewer
slots; I/O-bound steps can handle more.
Scale the fleet
Start with one worker, then scale up:
docker compose up -d
docker compose up -d --scale worker=5Verify enrollment:
swamp worker listEach worker appears with its auto-generated name and the fleet=compose-fleet
label.
To scale down, Docker sends SIGTERM to the excess containers. Each drains its in-flight work before exiting:
docker compose up -d --scale worker=2Run a parallel workload
Create a workflow with a forEach that fans work across the fleet:
jobs:
- name: parallel-work
forEach:
items: ["a", "b", "c", "d", "e"]
steps:
- name: process
task:
type: behavior
behavior: "processor-${{ self.item }}"
methodName: run
labels:
fleet: compose-fleetswamp workflow run fan-outSteps dispatch to available workers across the fleet. The orchestrator selects the worker with the most available capacity for each step.
Per-behavior lock: steps that write to the same behavior definition serialize on the definition's datastore lock. If the
forEachitems all target the samebehavior, they run sequentially regardless of fleet size. Use distinct behavior names (as above, with${{ self.item }}) for true parallelism. See Workflow Placement for scheduling details.
Add lifecycle policies
To run workers that process a fixed number of steps and exit:
environment:
SWAMP_WORKER_MAX_DISPATCHES: "10"To run workers that exit after 10 minutes of inactivity:
environment:
SWAMP_WORKER_IDLE_TIMEOUT: 10mBoth produce exit code 0, so restart: on-failure does not restart the
container.
Related
- Run a Worker in Docker — single-worker setup and TLS configuration
- Run a Fleet on Kubernetes — deploy to Kubernetes instead
- Scale from Zero Workers — submit work before workers connect
- Worker Commands — full CLI and environment variable reference