Exe Dev/swamp Serve Fleet
One-shot bootstrap for a swamp serve orchestrator + remote worker fleet on
exe.dev, run from an operator's local workstation. Creates the orchestrator
VM and workers, joins them to a tailnet, installs a pinned Deno + swamp,
mints the serve/worker/dashboard tokens into a vault, enables serve and the
workers as systemd user daemons, exposes serve through tailscale serve and
the exe.dev HTTPS proxy, promotes the operator's local swamp data into a
shared datastore, and — after an operator approval — claims the datastore for
serve (team mode).
This generalizes the reference deployment documented in the webframp
devsecops repo. Every site-specific value (names, admin principal,
datastore, vaults, ports) is an input with a generic default. No confidential
values are baked in.
Architecture (no reverse proxy)
Serve binds plain HTTP on loopback. Two native proxies terminate TLS on their
own side: the exe.dev HTTPS proxy fronts the browser dashboard; tailscale serve fronts the workers. Putting a reverse proxy in the middle forces Host
rewriting, Origin spoofing, CORS for the dashboard's crossorigin assets, and
hand-wired WebSocket upgrades — all avoided by binding plain and letting each
platform's proxy do TLS.
Prerequisites
- An exe.dev account and an API token in a vault (the
@webframp/exe-dev/vmsetupmethod mints it). - A tailnet with the exe.dev catalog integration configured, and a tailnet auth key in a vault.
- Credentials in the default chain for the chosen shared datastore (S3/GCS/...).
Phases
provision_orchestrator— orchestrator VM, serve daemon, proxies, tokens.enroll_worker— worker VMs enrolled under the placement label.verify—/ready+swamp worker list.promote— local data into the shared datastore; vault migration.- operator approval gate.
cutover—datastore claim+ restart serve (team mode).
Running
swamp extension pull @webframp/exe-dev/swamp-serve-fleet
swamp behavior create @webframp/exe-dev/swamp-serve-fleet swamp-serve-fleet
# edit the behavior and wire its global arguments:
# adminPrincipal, datastoreConfig, tokenVault,
# exeToken: ${{ vault.get("exe-dev", "API_TOKEN") }}
# tailnetAuthKey: ${{ vault.get("tailscale", "AUTH_KEY") }}
swamp workflow run @webframp/bootstrap-swamp-serve-fleet \
--input migrate_vaults='["gitlab"]' --input vault_type=@zts/aws-ssm
# the run suspends at the approval gate; approve to cut over:
swamp workflow approve @webframp/bootstrap-swamp-serve-fleet --run <id>See the bundled README for the full configuration shape and gotchas.
Release Notes
2026.11.05.1
Initial release.
Added
@webframp/exe-dev/swamp-serve-fleetprovisioner behavior with six methods:provision_orchestrator,enroll_worker,verify,promote,cutover, andteardown.@webframp/bootstrap-swamp-serve-fleetworkflow chaining the phases with a single operator approval gate before the irreversiblecutover(autoResume: true).- Composes
@webframp/exe-dev/vm(declared dependency) for VM lifecycle and drives swamp, tailscale, and exe.dev CLIs over SSH for serve/worker/token and promotion wiring. - Production serve flag set (plain HTTP on loopback,
--trusted-hostscovering every presented name, no middle reverse proxy) assembled generically from operator inputs. - Secret discipline: tokens minted during provisioning, copied into the
operator-named
tokenVaultvia a reveal→vault putpipeline, and delivered to workers over SSH stdin (never argv, never worker→orchestrator SSH); resources store token names and vault keys only, never values; captured SSH/CLI output is secret-redacted before persistence. - Serve and worker daemons are enabled as systemd user services
(
daemon enable --user), matching the reference deployment's lingered user-service model. verifyreads the server token from its vault key, asserts it is non-empty, and treats an unparseableswamp worker listas unhealthy rather than silently reporting zero workers.provision_orchestratorkeeps the exe.dev HTTPS proxy private (share set-private) so the dashboard is never left publicly reachable.provision_orchestratorandenroll_workerare idempotent: VM creation is skipped when a VM of that name already exists (the fleet is synced and checked first), token provisioning rotates/recreates an existing token with a fresh secret instead of failing on a duplicate name, and the remaining steps (tailscale join, installs, daemon enable, proxy wiring) are self-asserting. A re-run after a partial failure converges rather than erroring.
Security
adminPrincipal,tokenVault,datastoreType,magicDnsName, andclubApiKeyPathare format-constrained to reject shell metacharacters before they are interpolated into remote SSH (or localpromote) commands; a runtime-resolvedmagicDnsNameis validated before use.tailnetAuthKeyis format-validated (tskey-…) and delivered totailscale upover SSH stdin intoTS_AUTHKEY(sudo --preserve-env), so it never appears in argv or the remote/proctable.promoteandcutoverbuild theirswamp datastore/vaultcommands as explicit argv (nosh -c), so config blobs, vault names, and backend types are passed as individual arguments and cannot inject a command.cutovertolerates an already-claimed datastore on a re-run and always records its result: ifdatastore claimsucceeds but the serve restart fails, the resource is still written withserveRestarted: falseand arestartError, so a partial cutover is recoverable rather than leaving the store claimed, serve down, and no record.promoteacceptsmigrateVaultsas an array or a JSON-array string (the shape a workflow string input passes), parsing and re-validating each element against the vault-name constraint.- The token reveal→
vault putpipelines run underset -o pipefail, so ajqparse failure fails the step instead of silently piping an empty token intovault put(which would surface later as a misleading "empty value" error inenroll_worker). datastoreConfigis validated as a JSON object at the schema level, so a malformed blob is rejected early with a clear error rather than reachingswamp datastore setup.teardownchecks both VM-destroyrunModelresults (and passesmodelTypeexplicitly) instead of discarding them, so a failed destroy raises an error naming the still-running VMs rather than exiting clean while VMs keep billing.- The
swamp installsteps cap their remote output (pipefail+tail -c) so a verbose installer cannot grow the provisioner's capture buffer without bound. provision_orchestratorandenroll_workerpin swamp's bundled Deno todenoVersion(deno upgrade --version) after install, so the configured version is actually honored on the fleet rather than being metadata only.- The secret-stdin SSH helpers arm their SIGTERM/SIGKILL timeouts before writing stdin, so a remote process that exits without reading stdin cannot hang the call on a full pipe buffer.
| Argument | Type | Description |
|---|---|---|
| cpu? | number | Orchestrator VM CPU count. |
| memory? | string | Orchestrator VM memory. |
| disk? | string | Orchestrator VM disk. |
| Argument | Type | Description |
|---|---|---|
| cpu? | number | Worker VM CPU. |
| memory? | string | Worker VM memory. |
| disk? | string | Worker VM disk. |
Stand up a complete swamp serve orchestrator + worker fleet on exe.dev from the operator's local workstation, promote local data into the shared datastore, pause for an operator approval, then cut over to team mode (serve owns the datastore). The provisioner behavior `swamp-serve-fleet` (type @webframp/exe-dev/swamp-serve-fleet) must be pre-created with its global arguments wired (orchestratorName, adminPrincipal, datastoreConfig, tokenVault, exeToken, tailnetAuthKey, ...).
- Has README or module doc2/2earned
- README has a code example1/1earned
- README is substantive1/1earned
- Most symbols documented1/1earned
- No slow types (deprecated)1/1earned
- Dependencies pass trust audit2/2earned
- Docs keep secrets in a vault1/1earned
- Has description1/1earned
- Platform support declared (or universal)2/2earned
- License2/2earned
- Verified public repository2/2earned
Spawns a subprocess with Deno.Command().
- models/exe-dev/swamp_serve_fleet.ts:345 — no reason given
- models/exe-dev/swamp_serve_fleet.ts:450 — no reason given
- models/exe-dev/swamp_serve_fleet.ts:517 — no reason given
- models/exe-dev/swamp_serve_fleet.ts:1627 — no reason given
- models/exe-dev/swamp_serve_fleet.ts:1652 — no reason given