Skip to main content
← Back to list
01Issue
FeatureShippedSwamp CLI
Assigneesstack72

Relationships

#825 Surface extension type+method detail in CLI to eliminate expensive discovery loops

Opened by stack72 · 6/25/2026· Shipped 6/26/2026

Problem

In competitive benchmarking (swamp vs Terraform/Pulumi on identical AWS tasks), we identified a major efficiency gap: swamp agents spend ~10 tool calls and significant tokens on extension discovery before they can determine whether a needed capability exists. Competing tools pay zero discovery cost because their resource names and methods are predictable.

Benchmark scenario

Task: "Snapshot every EBS volume across 3 regions and tag them."

The agent followed the skill's discovery flow and made these calls:

  1. swamp model type search "ebs" — empty (no local types, clean repo)
  2. swamp model type search "snapshot" — empty
  3. swamp model type search "volume" — empty
  4. swamp model type search "aws" — empty
  5. swamp model type search "ec2" — empty
  6. swamp extension search ebs — fuzzy matches (efs, ecs), not ebs
  7. swamp extension search ec2 — found @swamp/aws/ec2
  8. swamp extension search aws — large result set
  9. swamp extension pull @swamp/aws/ec2 — installed 106 types
  10. swamp model type describe @swamp/aws/ec2/volume — discovered volume type has no create-snapshot method

Total wasted: ~10 tool calls, 0 useful output. The agent then fell back to command/shell.

A Terraform agent writes resource "aws_ebs_snapshot" directly — no discovery loop at all.

Root causes

1. The registry already has type+method data, but the CLI doesn't surface it.

The extension registry website (e.g. swamp-club.com/extensions/@swamp/aws/rds) shows every type and method an extension contains — including method signatures, arguments, and outputs. But swamp extension info @swamp/aws/ec2 only shows:

Contents:    "models"

No type names, no methods. The agent cannot inspect an extension's contents without pulling it first.

2. The skill's discovery flow doesn't mention swamp extension info.

The extension guide's "Before Creating an Extension" checklist goes: search → pull → describe. There's no step to inspect extension contents before pulling. The extension info command exists but isn't in the discovery flow.

3. No cold-repo shortcut in the skill.

When no extensions are installed (common in benchmarks and CI), every swamp model type search returns empty. The skill tells agents to "search local types first, then registry" without noting that local search is guaranteed-empty when nothing is installed. This caused 5 wasted calls.

4. The command/shell prohibition creates a dead end.

The skill says "Never use command/shell to wrap service integrations." But some operations (EBS snapshots, AMI creation, S3 object ops) have no extension type because CloudControl/CloudFormation doesn't model them. When the agent confirms no type exists, it has no sanctioned fallback. The rule needs a carve-out for operations where no extension type exists and building one isn't practical.

Proposed solution

Three changes: one CLI enhancement and two skill/docs updates.

1. Surface type+method metadata in swamp extension info (CLI change)

The registry API already returns contentNames and the website renders full type+method detail. The extension info CLI command and its renderer need to surface this data.

Requirements:

  • swamp extension info @swamp/aws/ec2 log output should show types and their methods (may need a --verbose flag given ec2 has 106 types)
  • swamp extension info @swamp/aws/ec2 --json must include the full type+method structure so agents can parse it programmatically
  • The data is already available from the registry API — this is a presentation layer change, possibly with a more detailed API call to fetch the per-type method metadata that the website already renders

With this change, the benchmark agent would run swamp extension info @swamp/aws/ec2 --json, see that ec2/volume only has get and sync methods (no create-snapshot), and fall back immediately — no pull needed.

2. Update the skill's discovery flow (docs change)

Update .claude/skills/swamp/references/extension/guide.md "Before Creating an Extension" section:

  • Insert swamp extension info <name> --json as a step between finding an extension via search and pulling it. Agents should inspect types and methods before pulling.
  • Add a cold-repo shortcut: "If no extensions are installed (e.g. fresh repo, CI environment), skip swamp model type search and go directly to swamp extension search — local search is guaranteed-empty."
  • Add a "confirmed no type/method exists" heuristic: "If extension info confirms no type or method covers the operation you need, stop searching and fall back to command/shell or a custom model. File a feature request (swamp issue feature --extension <name>) to track the gap."

3. Nuance the command/shell prohibition (docs change)

Update the skill's guidance on command/shell from "Never use command/shell to wrap service integrations" to something like:

"Don't use command/shell when an extension type covers the operation. If extension info confirms no type or method exists for the operation, command/shell or a custom model is the correct approach — file a feature request to track the gap."

Impact

These changes would reduce the benchmark discovery loop from ~10 tool calls to 2-3:

Step Before After
Find the extension 5 empty local searches + 2-3 extension searches 1 extension search (skip local — cold repo)
Check capabilities Pull 106 types + describe 1 extension info --json
Decide fallback Agonize over command/shell prohibition Skill says: confirmed no method → fall back

Scope

Changes needed in:

  • Extension info CLI command renderer to surface type+method data from the registry
  • Extension info JSON output to include the full type+method structure
  • Swamp skill extension guide (discovery flow, cold-repo shortcut, fallback heuristic)
  • Swamp skill model guide (command/shell rule nuance)

The registry API and website already have the data — no backend changes needed.

02Bog Flow
OPENTRIAGEDIN PROGRESSSHIPPED+ 1 MOREASSIGNED+ 7 MOREREVIEW+ 3 MOREPR_MERGED+ 1 MORENOTIFICATION_SKIPPED

Shipped

6/26/2026, 8:55:38 PM

Click a lifecycle step above to view its details.

03Sludge Pulse
stack72 assigned stack726/26/2026, 6:07:04 PM

Sign in to post a ripple.