Two researchers at SonarSource — the static-analysis company behind SonarQube — just published the cleanest answer I've seen to a question the benchmark industry keeps not asking: does the quality of the codebase itself change how a coding agent performs? Every agent evaluation you've read holds the codebase fixed and varies the model, the harness, the prompting. Priyansh Trivedi and Olivier Schmitt did the converse — same agent, same task, and a codebase that comes in two versions: one clean, one deliberately trashed.
The construction is the delightful part. You can't find this comparison in the wild, because no repository exists in both states at once. So they built minimal pairs: six pairs of repositories matched on architecture, dependencies, and external behavior, differing only on static-analysis violations and cognitive complexity. The pairs run in both directions — a pipeline charmingly named slopify degrades a clean repo (inlining helpers, duplicating logic across paths, padding files with dead code), and a vibeclean pipeline tidies a messy one. Then 33 tasks, hidden tests at the application's public surface, and 660 trials of Claude Code running Sonnet 4.6, blind to which side of the pair it was working on. It made Hacker News' front page this weekend, deservedly.
The agent succeeds either way — and that's the trap
The headline finding splits in two, and the split is where all the interest lives.
Pass rate: no change. The agent solved tasks in the sloppy codebase just as often as in the clean one. If your evaluation of "can the agent handle our repo" stops at task success — and nearly every evaluation does — cleanliness is invisible.
Operational footprint: substantial change. On cleaner code, the agent used 7 to 8% fewer tokens and revisited files it had already read about 34% less often. The work got done either way; the path through the work was very different.
That revisitation number is my favorite in the paper, because it's confusion made measurable. A file re-read is the agent going back to something it already looked at because the structure didn't hold in its working memory the first time. Humans do exactly this in a messy codebase — we just call it "wait, where does this actually happen?" and nobody meters it.
Which is the durable point. Tech debt was always expensive — the expense just never appeared anywhere legible. It got paid in senior-engineer archaeology, in onboarding weeks, in the low-grade tax of everyone navigating a 2,800-line class — costs that are real but appear on no invoice, which is precisely why the refactor never got budgeted. Token-metered agents change the accounting. The same navigational confusion now lands as input tokens, at a price per million, on a bill with your team's name on it. It's the taxi meter problem: a cab driver in a badly signed city still gets you to the address — you simply pay for every wrong turn, and the meter doesn't distinguish distance from disorientation.
The caveats, honestly
Three things to hold alongside the finding.
The vendor interest is right there in the byline. SonarSource sells static analysis, and this study concludes that static-analysis cleanliness matters. To the authors' credit, the design works hard for its independence: bidirectional pair construction, hidden tests, a blind agent, and — the part that impressed me — a comment-volume ablation. Cleaner code can carry more comment text (one pair's clean side had over 8,000 more comment lines), which could have masked or manufactured the effect. When they normalized comments, the cleaner side got cheaper still on three of the four ablated pairs — the structural effect had been partially hidden under docstring bulk, not created by it. They also state the limitation plainly: they selected the repos, built both sides, and authored every task themselves. Careful, but self-curated end to end.
One agent, one model. All reported numbers come from Claude Code on Sonnet 4.6. They swept Haiku 4.5 too, but its pass rate was too low to read footprint differences cleanly. Whether the effect holds across other harnesses is exactly the kind of question the schema-fidelity finding from yesterday should make you refuse to assume.
7 to 8% is modest per run. Nobody's retiring on a single-digit token saving. The compounding is where it bites: the paper cites a survey finding agent activity in roughly 22 to 29% of 128,000 GitHub projects surveyed, and prior work putting a single SWE-bench Verified task at around 4 million tokens on average. Single digits of a number that large, on every run, forever, is a line item.
The loop that should worry you
Here's the connection the paper gestures at that I'd underline twice. Look again at what slopify does to degrade a repo: inline helpers into callers, duplicate logic across paths, pad files with dead code. That list should sound familiar — it's a fair description of what unreviewed agent output drifts toward over many iterations. Which means the mess agents make is the mess the next agent run pays to navigate. Sloppy generation this sprint quietly raises the token cost of every future sprint in the same repo. Tech debt always compounded; now the interest is denominated in a currency your finance team can see.
What I'd actually do
Re-run the refactor math. The cleanup you couldn't justify on human-productivity grounds ("it works, don't touch it") may now have a computable dollar case. If agents work in a repo daily, a structural cleanup is an investment with a measurable token yield. This argument is for teams with real agent volume — if agents touch your codebase twice a month, the math won't move.
Watch footprint, not just pass rate. If you evaluate agents against your own codebase, log tokens and file re-reads per task alongside success. Pass rate told this study nothing; the meter told the whole story.
Gate the slop at generation time. Since agent-produced mess taxes future agent runs, the linter check on agent output stops being pedantry and becomes cost control — a deterministic check, cheap to run on every PR, no judgment required.
The forward question I'm left with: benchmarks taught us to ask whether the agent can do the task. Now that we're billed by the token, the better question is what your codebase makes the task cost — and that number, unlike the leaderboard, is one you can actually change this quarter.