2026年7月7日 星期二

[AI Insights] Why Superpowers 6 Cuts AI Coding Costs

 [AI Insights] Why Superpowers 6 Cuts AI Coding Costs

Summary: Superpowers 6 shows that AI coding costs often come less from inference itself and more from repeatedly re-reading the same context.




Content:


Many people assume AI programming is expensive because the model itself is expensive. Superpowers 6 offers a very different answer. In many cases, the biggest cost does not come from a few inference calls, but from making AI repeatedly read the same context.


For example, the same Git diff may be read once by a Diff Coordinator, again by an Implementer, and then reviewed again by a Reviewer. Likewise, a Plan Controller may break down a task, only for a Subagent to reread everything from scratch during execution. For human developers, this kind of repetition is almost free. For LLMs, every repeated read becomes tokens, and once that context is included, later steps keep paying for it.


That is why this video focuses on a counterintuitive optimization. Superpowers 6 does not mainly save money by downgrading every step to cheaper models. Instead, it reduces waste by making agents read less irrelevant material, avoid repeatedly fetching the same information, and delegate deterministic work to scripts. Expensive context is then reserved for actual judgment.


On June 9, 2026, Anthropic released Fable 5. On June 12, access to Fable 5 and Mythos 5 was suspended due to U.S. export control requirements. Anthropic later explained that the practical effect of those restrictions was that access had to be disabled for all users. On June 30, the export controls were lifted, and on July 1, Fable 5 access began to return.


The optimization work Jesse Vincent did for Superpowers 6 took place during Fable 5’s early availability window. Across 25 experiments over 36 hours, the listed cost was about $165. At unsubsidized token pricing, the cost would have been around $650.


According to the author’s blog, on the Anthropic Evo benchmark, Superpowers Build reduced wall-clock runtime by about 50% and token spend by about 60%. However, an important boundary must be added here: this 60% figure reflects the author’s highest-end Evo result and should not be assumed to reproduce across every project, harness, or workload.


The wording in the Superpowers V6.0 Release Notes is more conservative. It says that in their Evo runs, Cloud Code and Codex had similar quality, but were about twice as fast and used close to 50% fewer tokens. This actually makes the optimization more credible, because it includes failed cases, risk records, and rollback decisions rather than just presenting a large savings number.


If you have never used Superpowers, here is a quick background. Superpowers can be understood as a software development methodology designed for coding agents. It does not just ask AI to write code. It also constrains how AI understands requirements, breaks down work, writes tests, and performs reviews.


A typical flow looks like this:


First comes Brainstorming. The agent does not immediately jump into editing code. Instead, it asks about goals, boundaries, risks, and acceptance criteria, then produces a design document.


Next is the isolated workspace stage. Using Git Worktrees, the system checks whether you are already in an isolated environment. If not, it creates or guides the creation of a Git worktree, runs initialization scripts and baseline tests, and then writes a plan.


This plan is not just a few sentences about implementing a feature. It includes file paths, interfaces, test commands, and expected outcomes. Each task must be independently testable and independently reviewable. Within each task, each step is meant to be only a 2–5 minute action.


Execution then moves into Subagent Driven Development. The Controller assigns each task to a new Subagent. That Subagent receives only the context it needs, follows TDD through Red, Green, and Refactor, commits the code, and then passes it to a Reviewer.


Per-task review became especially important in V6. After each task, a Reviewer returns two verdicts at once: Spec Compliance and Code Quality. In other words, did the implementation satisfy the requirement, and is the code itself well written? Both are judged in the same review round.


After all tasks are completed, there is also a Whole Branch Review, followed by Finishing A Development Branch. At that point, the system runs full tests and decides whether to merge, open a PR, keep the branch, or discard it.


This process is stable, but before V6 it also had clear tradeoffs: high cost and slow execution. Jesse wrote that the most common user complaint was token expense. Superpowers also felt slow at scale. While slowness can be tolerated if things run automatically in the background, both slowness and cost hurt the user experience.


The first major optimization targeted the handoff from Coordinator to Reviewer. Fable analyzed a large number of Subagent Driven Development sessions and found that Code Reviewers and Spec Compliance Reviewers often ran many Git commands during review. That sounds normal, because a Reviewer wants to inspect what changed and may call commands like git log, git diff, and git stat.


The problem is that command output in an LLM workflow is not free. Once a diff is expanded, the output enters the observation stream and can be repeatedly carried through later turns. A medium-sized refactor diff can easily consume tens of thousands of tokens. Worse, that information has often already been processed by other roles. The Coordinator saw the context while planning, the Implementer saw the relevant files while coding, and the Reviewer then fetches it all again.


V6 solved this in a simple but effective way. Before the Reviewer starts, a script generates a Review Package. This package includes the commit list, file change statistics, and the full diff with context. The Reviewer receives a file path instead of needing to run Git commands manually.


This Reviewer handoff redesign delivered about a 10% improvement in both token usage and wall-clock time. A similar idea appears in the Task Brief. Instead of repeatedly giving the full plan to every Subagent, the Controller extracts the current task into a brief file. The Subagent reads its own brief rather than searching through a long plan.


A useful comparison is this: the old flow was like sending the Reviewer into the warehouse to gather evidence alone. The new flow is like preparing an evidence bag before work begins. The Reviewer only needs to read the bag. The change is not flashy, but it is deeply practical. If information can be computed reliably by scripts, it should not be fetched live through an LLM context window.


The second major optimization was reviewer consolidation. In V5, each task had two separate reviewers: one for Spec Compliance and one for Code Quality. That sounds safer on paper, but experiments showed the design was expensive and overly sensitive to how the Controller prompted the system.


V6 merged the two roles into a single Task Reviewer prompt. That Reviewer reads the diff once and returns both verdicts together: whether the requirement was implemented correctly and whether the code can be trusted.


An interesting new status was also introduced: Cannot Verify From Diff. Some requirements are not visible in the current diff or span multiple tasks. In the past, a Reviewer might try to reconstruct the full situation by aggressively exploring files and history, which consumed tokens and weakened task boundaries.


In V6, the Reviewer is allowed to explicitly admit that it cannot verify something. If a requirement cannot be validated from the current diff, it is marked as a warning and escalated to the Controller. Since the Controller already has the full plan and cross-task context, it is better suited to make that judgment.


This should not be misunderstood as “less context is always better.” In one of the Superpowers Evo experiment sets, this exact mistake appeared. If the Reviewer received only the Diff Package but not the Task Brief, it tended to confidently reduce the spec into vague global constraints. In five recorded runs, it never once identified the missing brief. This is a valuable finding: saving tokens is not about blindly deleting context. It is about removing duplicated information while preserving task definitions, interfaces, tests, and constraints.


The third optimization was model layering. Many teams instinctively send every AI coding task to the strongest model available. The reasoning is simple: if quality matters, then use the most powerful model everywhere.


Superpowers 6 suggests a more nuanced strategy. Mechanical tasks can often use cheaper models. For example, if the plan already provides complete code guidance and the Implementer mainly needs to transcribe, run tests, and fix small issues, a lower-cost model may be enough. But tasks that require reading prose plans, doing multi-step reasoning, or judging architectural boundaries should not be downgraded casually.


The same applies to reviewing. Small diffs and mechanical edits may be handled by cheaper models. But anything involving concurrency, shared state, cross-file contracts, or model-level architectural judgment requires a stronger model.


V6 also enforced an explicit model declaration on every dispatch. Without that, many harnesses would inherit the most expensive model from the current session by default. The release notes mention one case where 26 reviewers all ran on the top-tier model. That is how money quietly disappears.


The experiments also tested several ideas that sounded reasonable but performed poorly.


The first was limiting Controller thinking. The logic seemed straightforward: if reasoning is expensive, cap the Controller’s thinking budget. But the result backfired. The average number of turns rose from 92 to 138, and output volume increased as well. Thinking less early on caused more wandering and more repair cycles later. Any tokens saved upfront were quickly lost in follow-up corrections.


The second was shrinking plan length. Many people assume long plans must be wasting tokens. In the experiments, a plan word budget was added, and even code sections were exempted, yet test signals still dropped by 62%. What got removed were the very anchors needed later: test definitions, interfaces, and task structure. This result is highly relevant for engineering teams. The value of a plan lies in the guidance it provides for execution and review. If those anchors are removed, each agent has to guess for itself.


The third was using Sonnet to generate the plan. Sonnet is cheaper and often performs well, so why not let it write the plan? The E11 result was subtle. Sonnet did not obviously lose requirements, global constraints, or interface fidelity. The problem was task structure. Four out of five plans collapsed into too few tasks. The average task count dropped from 5.8 under Opus to 3.6.


The visible part of the plan may have looked acceptable, but the execution structure underneath became less effective. That kind of degradation is easy to miss if you only compare final summaries rather than how work is decomposed.


Overall, the most important lesson from Superpowers 6 is that AI coding optimization is not mainly about making every step cheaper. It is about reducing repeated context loading, preserving the right task anchors, and assigning deterministic work to scripts instead of language models.


In short, the expensive part is often not the model call itself. The expensive part is making the model repeatedly reread the same information. Superpowers 6 reduced cost by changing workflow design, not just by swapping models. That is what makes this case especially worth studying.

沒有留言:

張貼留言