Skip to main content
← Back to list
01Issue
BugShippedExtensionsPublic
Assigneesskunk-ape

Relationships

#4755 Swampify: compare counts model steps, fixes and the latest run, and the skill records built only for output the user accepts

Opened by skunk-ape · 11/14/2026· Shipped 11/15/2026

Found in an external tester's Swampify run, from a swamp start kick-off through a built workflow and compare. Builds on #4706 (compare and the compared state) and #4714 (the hand-off to the page).

Repo: swamp-extensions (@swamp/agent-sessions: compare and the swampify skill).

Goal

The comparison tells the truth about what a built option costs, and the skill records built only for output the user accepts.

What happened

  • The tester built a workflow that calls a model in some steps. Its comparison showed zero tokens for the swamp run, though a later run of it used about 54,000 input and 1,800 output tokens. The break-even came out at 5 occurrences, which their agent called optimistic.
  • Every chore minimum in the history was 0 (tokens, tool calls, active time), and the chore median equalled the wholeTurns median on every field, so break-even low and high were the same.
  • The first run's output was wrong, and the skill recorded built anyway. A second session fixed the workflow and re-ran it. compare still counted only the first session as the build, and its swamp-run figures still described the first run.
  • The handoff's confirm-the-target step asked "which backlog should it use?", offering the brand-new workspace (which has none) as an option. The tester didn't understand the question.
  • The disclosure was given on the start page and again, in full, by the skill.

Changes

compare

  • Swamp run tokens. Sum the model tokens of every step in the target's run that called a model (for example @swamp/agent-runner steps), from the run's own records. A run with no such steps stays at zero. If a step's usage isn't recorded, say so in the comparison rather than reporting zero.
  • The latest run, too. Keep the first run, and add the latest successful run when it differs, so a fixed target shows what it costs now.
  • Build cost includes fixes. Count, as build cost, later sessions that changed the built target's definition (found from session records: calls that edited or re-created it, or ran built again), and say how many sessions it took.
  • Chore attribution. Find out why one occurrence attributed no calls and why the chore medians match wholeTurns exactly. Fix it, or document why. An occurrence with no attributed calls is reported, not counted as zero.
  • Break-even uses the swamp run's real tokens. It is null when the swamp run costs as much as the chore by hand, as #4706 specified.

The skill

  • Before built. After the first run, show the output and ask the user whether it's right. Record built only after they agree. If they don't, fix and re-run in this session.
  • The handoff's target question. Ask only about concrete choices that make sense for this workspace and this option: name the real systems the target would read or write, say which are read-only, and never offer an empty new workspace as a source. If there is nothing to choose, don't ask.
  • The disclosure after a kick-off. The start page already gave it. Restate it in one line with the window and scope, and still wait for agreement.

Out of scope

  • The start page and launch (swamp).
  • Replay.

Done when

  • compare unit tests cover: a workflow with model steps, a behavior run with none, a step with no recorded usage, first and latest runs, build cost across a fixing session, an occurrence with no attributed calls, and break-even against real swamp tokens, including null.
  • The property test from #4706 still holds (chore never exceeds wholeTurns), and a new one shows break-even never goes down when the swamp run costs more.
  • Routing evals: built waits for the user's confirmation; the target question isn't asked when there is nothing to choose; a kick-off gets the one-line disclosure.
  • The skill passes tessl review at 90% or above, and its trigger and routing evals still pass.
  • A manual run builds a workflow with a model step, fixes it once, and compare shows the model tokens, both runs and both build sessions. No transcript content goes in the PR.
02Bog Flow
✓OPEN✓TRIAGED✓IN PROGRESS✓SHIPPED+ 1 MOREASSIGNED+ 5 MOREREVIEW+ 7 MOREPR_MERGED+ 2 MORESESSION_SUMMARIZED

Shipped

11/15/2026, 2:18:24 AM

Click a lifecycle step above to view its details.

03Sludge Pulse
skunk-ape assigned skunk-ape11/14/2026, 6:47:38 PM
Editable. Press Enter to edit.

system commented 11/14/2026, 6:41:52 PM

Classified automatically when this issue was filed.

  • Source: Extensions

If you feel this classification is incorrect, add a ripple to tell us so.

Sign in to post a ripple.