Code Review that uses 8x fewer tokens, 2x as fast

The Challenge

David Cramer of Sentry wrote a post titled “We Cant Afford Your Inference”. In it, he describes spending more than $10,000 on tokens in a single week, across multiple models and workloads. His conclusion was that new models are too expensive to use broadly at Sentry.

He attributed a significant part of that spend to how the models ran his Garfield skill, an adaptive code review framework that makes sure code follows the policy and standards of its repository after an agent edits it. Garfield reviews the code, repairs what it finds, and verifies the result.

The Swamp team took the post as a challenge: could Swamp change the economics of Garfield while keeping its behavior intact?

Garfield's reviews were good. It inspects the repository it runs in, discovers the policy specific to that codebase, and uses the model's own judgment to evaluate the code, re-evaluate its own work, and decide when it is done. The cost was the problem. Run in Codex on a representative change, the skill used about 4.5 million tokens, ran for about 12 minutes, and dispatched 23 sub-agents.

The skill relies on a coordinator agent. The coordinator dispatches a rolling series of review sub-agents, collects their verdicts, adjudicates the findings, and then runs a final verification pass for tests, formatting, and lint. Every one of those steps runs through the model, including the bookkeeping that ordinary code has handled reliably for decades.

The Swamp team identified 3 areas they wanted to improve:

• Cost: reduce the tokens spent on each review without lowering its quality
• Speed: shorten the time an engineer waits for a review to finish
• Visibility: see what each reviewer found, and why the loop decided it was done

The Solution

A Swamp workflow that performs the same review, repair, and verify loop as the skill. The coordination that the skill handed to an agent became deterministic code in a single Swamp behavior. The behavior calls an agent only for the parts of the job that need judgment, such as reviewing a change against the repository's policy or repairing a finding, and checks their results in code.

On the same change, the workflow used about 500 thousand tokens, ran for about 6.5 minutes, and used 3 agents.

How it works

The Swamp team began by reading the skill from start to finish and taking notes on its structure. They then asked Codex to explain it:

Read the garfield skill and summarize for me what it does, and how you are likely to interpret its instructions in practice. Draw ASCII diagrams to help me understand.

The summary confirmed their reading. Garfield is a review loop for code that has already been written, and it adjusts that code to fit the style and architecture of the codebase rather than proposing large refactors. Asking for the summary also put that understanding into the agent's context before the next step.

Next, they asked Codex to translate the skill into a Swamp extension:

Translate the garfield skill into a Swamp extension. The goal is to translate the behavior of the skill as directly as possible into deterministic code within Swamp behaviors and workflows.

The resulting behavior stores the output of each sub-agent as versioned, typed data. Every step of a review can be inspected afterward, and sub-agents can see the history of their own earlier work, which avoids repeated rework cycles. The Swamp team also added a risk review phase that scales the intensity of a review to the risk of the change. Most of the savings came from removing the coordinator agent.

To compare the two approaches, the Swamp team had the agent write a black box user acceptance test suite that runs both against real tasks:

Write a black box user acceptance test that runs both the existing skill and the new Swamp workflow. Measure their performance, token usage, and execution time. The test suite should report their success or failure to perform real world tasks, and tell me the level of improvement between the existing skill and the Swamp workflow.

The agent produced the benchmarking suite, a test codebase to run it against, and the tests themselves. The extension, the benchmarking suite, and the full results are in the Garfield extension repository.

Benefits

The resulting workflow delivered on every area the Swamp team hoped for:

• Cost: an 8x reduction in tokens per review, from about 4.5 million to about 500 thousand.
• Speed: reviews finished in about half the time, from about 12 minutes to about 6.5 minutes.
• Visibility: each reviewer's findings are stored as versioned, typed data, so the Swamp team can see what was found at every step and why the loop stopped.

The benchmark suite recorded the following results:

On the harder payment-idempotency case, the workflow used 7.8x fewer tokens and finished almost 3x faster.

Lessons Learned

Any skill that encodes a complex process to follow should be converted to a Swamp behavior. When building the replacement, measure your improvement by building a black box testing environment.

Conclusion

Moving the deterministic parts of an agent skill into a Swamp workflow cut the token usage of code review by 8x and halved its run time, with the same review behavior. The Swamp team used the agent to build the program that reduces how much the agent is needed, and spends its tokens where the model's judgment adds the most value.