← Back to list
01Issue
BugShippedExtensionsPublic
Assigneesskunk-ape
Relationships
#4755 Swampify: compare counts model steps, fixes and the latest run, and the skill records built only for output the user accepts
Opened by skunk-ape · 11/14/2026· Shipped 11/15/2026
Found in an external tester's Swampify run, from a swamp start kick-off through a built workflow and compare. Builds on #4706 (compare and the compared state) and #4714 (the hand-off to the page).
Repo: swamp-extensions (@swamp/agent-sessions: compare and the swampify skill).
Goal
The comparison tells the truth about what a built option costs, and the skill records built only for output the user accepts.
What happened
- The tester built a workflow that calls a model in some steps. Its comparison showed zero tokens for the swamp run, though a later run of it used about 54,000 input and 1,800 output tokens. The break-even came out at 5 occurrences, which their agent called optimistic.
- Every
choreminimum in the history was 0 (tokens, tool calls, active time), and thechoremedian equalled thewholeTurnsmedian on every field, so break-evenlowandhighwere the same. - The first run's output was wrong, and the skill recorded
builtanyway. A second session fixed the workflow and re-ran it.comparestill counted only the first session as the build, and its swamp-run figures still described the first run. - The handoff's confirm-the-target step asked "which backlog should it use?", offering the brand-new workspace (which has none) as an option. The tester didn't understand the question.
- The disclosure was given on the start page and again, in full, by the skill.
Changes
compare
- Swamp run tokens. Sum the model tokens of every step in the target's run that called a model (for example
@swamp/agent-runnersteps), from the run's own records. A run with no such steps stays at zero. If a step's usage isn't recorded, say so in the comparison rather than reporting zero. - The latest run, too. Keep the first run, and add the latest successful run when it differs, so a fixed target shows what it costs now.
- Build cost includes fixes. Count, as build cost, later sessions that changed the built target's definition (found from session records: calls that edited or re-created it, or ran
builtagain), and say how many sessions it took. - Chore attribution. Find out why one occurrence attributed no calls and why the
choremedians matchwholeTurnsexactly. Fix it, or document why. An occurrence with no attributed calls is reported, not counted as zero. - Break-even uses the swamp run's real tokens. It is
nullwhen the swamp run costs as much as the chore by hand, as #4706 specified.
The skill
- Before
built. After the first run, show the output and ask the user whether it's right. Recordbuiltonly after they agree. If they don't, fix and re-run in this session. - The handoff's target question. Ask only about concrete choices that make sense for this workspace and this option: name the real systems the target would read or write, say which are read-only, and never offer an empty new workspace as a source. If there is nothing to choose, don't ask.
- The disclosure after a kick-off. The start page already gave it. Restate it in one line with the window and scope, and still wait for agreement.
Out of scope
- The start page and launch (swamp).
- Replay.
Done when
compareunit tests cover: a workflow with model steps, a behavior run with none, a step with no recorded usage, first and latest runs, build cost across a fixing session, an occurrence with no attributed calls, and break-even against real swamp tokens, includingnull.- The property test from #4706 still holds (
chorenever exceedswholeTurns), and a new one shows break-even never goes down when the swamp run costs more. - Routing evals:
builtwaits for the user's confirmation; the target question isn't asked when there is nothing to choose; a kick-off gets the one-line disclosure. - The skill passes
tessl reviewat 90% or above, and its trigger and routing evals still pass. - A manual run builds a workflow with a model step, fixes it once, and
compareshows the model tokens, both runs and both build sessions. No transcript content goes in the PR.
02Bog Flow
Shipped
Click a lifecycle step above to view its details.
03Sludge Pulse
Editable. Press Enter to edit.
system commented 11/14/2026, 6:41:52 PM
Classified automatically when this issue was filed.
- Source: Extensions
If you feel this classification is incorrect, add a ripple to tell us so.
Sign in to post a ripple.