HOW SWAMP WORKS

Swamp is an adaptive automation framework designed to be operated by AI agents. It decomposes automation into a small set of composable primitives: models, workflows, vaults, extensions, skills, and a data layer — also known as The Swamp. Each primitive does one thing. What matters is how they fit together.

Models are the unit of work

A model represents a typed interaction with an external system. It has a type (like command/shell or @swamp/aws/ec2/vpc), a set of methods (like create, sync, delete), and typed schemas for both its inputs and outputs. The model's type defines the shape of data it accepts and produces. A model definition is a YAML file that fills in the specifics: which VPC CIDR block, which shell command, which API endpoint.

This separation matters. The model type is reusable logic written in TypeScript. The model definition is configuration written in YAML. Because the definition is plain data rather than code, an agent can produce one using only its general reasoning ability. It writes structured YAML that conforms to the type's schema, and the type handles execution.

Every method execution produces versioned, immutable data artifacts. A command/shell model's execute method writes a result resource containing stdout, stderr, and exit code. A cloud model's create method writes the resource's state. This data is stored in .swamp, versioned automatically, and queryable. Any subsequent operation can look up what previous operations produced.

Workflows wire models together

A workflow is a DAG of jobs, where each job contains steps that run model methods. Jobs execute in dependency order. Steps within a job run in parallel.

Workflows become useful through data chaining. A step can reference the output of a previous step using CEL expressions. If step A runs a shell command that looks up the latest AMI, step B can read that AMI ID from step A's output and pass it as an argument to an EC2 model's create method. The expression data.latest("ami-lookup", "result").attributes.stdout reaches into the stored data from step A and extracts what step B needs.

This means workflows don't just sequence actions -- they compose data flows. The output of one model becomes the input of another through typed, validated references. If the data doesn't exist yet (because a dependency hasn't run), the expression fails loudly rather than passing through a blank value.

Workflows can also nest. A step can invoke another workflow instead of a model method, passing inputs down. This lets you build higher-order automation: a provisioning workflow that calls a networking workflow that calls individual resource workflows.

A workflow can also pause for a human. A manual_approval step suspends the run until someone approves it — see Understanding Workflow Suspension for why, and the workflows reference for the lifecycle.

Steps run locally by default, but they can also run on remote machines. A step with a target or labels field is dispatched to a connected worker instead of executing in the orchestrator process. The worker dials home, receives the step, runs it, and streams results back. From the workflow's perspective, the step behaves identically — data chaining, vault references, and output recording all work the same way. See Remote Execution for how the orchestrator-worker model works.

Vaults keep secrets out of the data path

The data flow described above has a problem: some values are sensitive, and they shouldn't be stored in plaintext YAML or versioned alongside the data they help produce. Vaults provide a separate path for secrets. Model definitions and workflows reference secrets by name rather than by value, and Swamp resolves those references at runtime -- not when the definition is created or the workflow is planned, but when the step actually executes. This means secrets are never frozen into YAML files, never written to .swamp data, and never cached between runs. The same principle applies to output: when a model's schema marks a field as sensitive, Swamp redirects that value into vault storage automatically, so secrets never persist in the data layer either.

There's a design tension between solo developers, who need secrets to work without external infrastructure, and teams, who need secrets to live in managed systems (AWS Secrets Manager, 1Password, and the like). Vaults resolve this through swappable providers: a local encryption provider works out of the box, while provider-backed vaults delegate to external secret management. The vault configuration lives in the repo, but the secrets themselves live wherever the provider puts them. Because a vault provider is one of the things an extension can package, teams can integrate with whatever secret backend they already use.

Extensions package reusable model types

When no built-in model type exists for a task, someone writes one. An extension model is a TypeScript file that exports a type definition with Zod schemas for inputs and outputs, and execute functions for each method. The file lives in extensions/models/ and Swamp discovers it at startup.

Extensions aren't limited to models. They can also package vault providers (custom secret backends), datastores (custom backends for .swamp data), and reports. A single extension manifest bundles whichever of these it needs, versioned with CalVer, and published to a registry.

The registry exists so that no single person or team needs to build every integration. Someone writes a model type for their own use, publishes it, and anyone else can install it. Trusted collectives go further -- referencing an @swamp/aws model type causes Swamp to resolve and install the extension automatically. Trust is deliberately narrow: only the first-party swamp collective is trusted by default, and any other collective must be trusted explicitly. Belonging to a collective grants the ability to publish its extensions, not consent to install them automatically -- so membership alone does not make a collective trusted.

Skills teach agents how to use Swamp

None of this works if the agent doesn't know how to use it. Skills are the bridge between Swamp's capabilities and an agent's ability to exercise them.

A skill is a markdown document -- not executable code -- that teaches an agent how to work with swamp. Early versions of Swamp shipped a separate skill for each topic area: one for models, one for workflows, one for vault configuration, and so on. This meant the agent had to figure out which skill to load before it could start working, and the skills themselves carried duplicated context about how the pieces fit together.

Swamp now consolidates all of that into a single swamp skill that acts as a gateway. When a user says "create a workflow," the agent loads the swamp skill, which routes to the relevant guidance internally -- the right CLI commands, the right YAML structure, the validation steps, the common pitfalls. The agent doesn't need to know the internal structure; it just loads one skill and gets the knowledge it needs. A second skill, swamp-getting-started, handles onboarding for new users as a dedicated path.

Skills are bundled with Swamp and live in the agent's workspace. Rather than building agent logic into Swamp itself, Swamp teaches the agent through documentation. The agent uses its general reasoning abilities to apply the skill's instructions to the user's specific situation. Because the interface is markdown rather than a plugin API, any agent that can read and follow instructions can use them.

Data querying ties it all together

Every model execution, every workflow run, every step produces data. Swamp stores this data with rich metadata: which model produced it, which spec it belongs to, tags from the model and workflow, a version number, a timestamp.

The data query system uses CEL (Common Expression Language) to filter across all stored data. You can find all failed results (attributes.status == "failed"), all data from a specific workflow (tags.workflow == "deploy"), or all resources larger than a megabyte (size > 1048576). Projection with --select extracts specific fields from matching records.

Data querying serves two audiences. For humans, it's an inspection tool -- what did the last run produce? What's the current state of my infrastructure? For agents and models, it's a composition mechanism. Extension models can call context.queryData() from within their execute functions to find and process data from other models. Models can also invoke other models directly via context.runModel() when they need to orchestrate sub-model execution and act on the result mid-method. Workflow expressions can use data.query() to conditionally branch based on what exists in the datastore.

The data layer also manages lifecycle. Resources have lifetimes (ephemeral, job, workflow, duration, infinite) and garbage collection policies. Old versions get cleaned up automatically. Data can be renamed with forward references so that refactoring names doesn't break existing workflows. The system is designed for data to accumulate and be useful, not to become debt.

The composition

These primitives compose in a specific pattern:

1. A skill teaches an agent what Swamp can do and how to do it
2. The agent creates model definitions from existing types, or writes extension models when no type exists
3. The agent wires models into workflows with data chaining expressions
4. Vault expressions supply secrets at runtime, keeping sensitive values out of configuration files and stored data
5. Each execution produces versioned data that's queryable and referenceable
6. Data queries feed into future decisions -- by humans inspecting results, by agents reasoning about state, or by models reading each other's output
7. When a model needs to orchestrate another model's execution mid-method, context.runModel() enables direct model-to-model invocation — bounded by dependency declarations, depth limits, and vault isolation (see Models, Types, and Methods)

Several properties of the system support this composition. Model definitions and workflows are YAML files committed to git, so they're reviewable and versioned alongside the code they automate. Data is immutable and versioned, so you can always see what happened. Schemas are validated at creation time and execution time, so type mismatches surface early. Extensions are published and versioned, so teams can share and build on each other's work.

Swamp is an architecture for adaptive, AI driven automation. It provides typed building blocks and a data layer. The agent decides how to assemble them for a given task.

The run tracker

Models and workflows produce data, but that data only appears after an execution finishes. While a method is running — or while a workflow is waiting on a dependency — there was no way to answer a basic question: what is executing right now? swamp model method history shows completed runs. Audit shows issued commands. Neither tracks in-flight work.

The run tracker fills this gap. Every model method execution and every workflow run registers with a local SQLite database (.swamp/run_tracker.db). Each registered run heartbeats every 30 seconds. If a heartbeat stops and the originating process is no longer alive, the run is considered stale and eligible for reaping. swamp run history lists recent and in-flight runs; swamp run doctor identifies and optionally clears stale ones.

The tracker uses SQLite rather than the shared datastore because run tracking is inherently local. A tracked run records a PID — a process ID that is meaningful only on the machine where the process is executing. PID 12345 on machine A is a different process than PID 12345 on machine B. Sharing this data across machines would create false signals. For remote visibility, swamp run history --server queries the server's own local tracker over WebSocket, keeping the data authoritative for each machine.

Runs are retained for 7 days and pruned automatically. The tracker is an operational tool — a window into what is happening right now — not an audit log or a durable record. The audit system, model method history, and workflow history serve those longer-lived needs.

Configuration

The primitives above need configuration to do useful work, and that configuration comes from four sources: .swamp.yaml (repo-level, checked into version control), swamp config set (user-level, stored in ~/.config/swamp/), environment variables prefixed with SWAMP_, and CLI flags. Precedence follows a strict order: CLI flags override environment variables, which override user config, which overrides repo config, which overrides built-in defaults.

This layering exists because different decisions belong to different scopes.

Repo config (.swamp.yaml) holds settings the whole team shares: which model types to trust, auto-GC policies, datastore backend selection. These belong in version control because they define how the repo behaves -- changing the trusted collective list or the garbage collection window affects everyone who works in the repo. See the repository configuration reference for the full .swamp.yaml schema.

User config holds personal preferences: log verbosity, telemetry opt-out, default output format. These should not be version-controlled because one developer's preference for debug-level logging shouldn't force that on the rest of the team.

Environment variables provide the override layer for CI/CD pipelines and scripted environments. Swamp uses the SWAMP_ prefix convention -- variables like SWAMP_REPO_DIR, SWAMP_LOG_LEVEL, SWAMP_NO_TELEMETRY, and SWAMP_DATASTORE -- so there is no ambiguity about which env vars belong to Swamp versus the surrounding toolchain. Env vars always win over both config files, which means a CI pipeline can force specific settings without forking the repo's .swamp.yaml or modifying user config on ephemeral runners.

CLI flags sit at the top of the precedence chain for one-off overrides: a developer running a single command with --log-level debug shouldn't need to change any persisted configuration.

The tradeoff this design makes is explicitness over convenience. Rather than merging configuration from implicit locations, each layer has a clear scope and a predictable override relationship. When something behaves unexpectedly, the precedence order tells you exactly where to look.