Why AI Ignores Messy Code but Your Token Bill Doesn’t

Experimental Setup

Researchers built six minimal‑pair repositories (three Java, three Python). Each pair has identical architecture, dependencies and test outcomes; the only difference is code cleanliness measured by SonarQube rule violations and cognitive complexity.

Slopify (degradation pipeline) : start from a clean repo, iteratively introduce violations (inline functions, duplicate logic, dead code) while keeping tests passing.

Vibeclean (refinement pipeline) : start from a naturally messy repo, iteratively remove violations (duplicate strings, dead code, outdated syntax, split large switch statements into helper functions).

The six pairs include public projects (Apache Commons BCEL, Netflix Genie, CKAN) and private SonarSource repos to avoid model memorization. On these, 33 tasks were manually crafted with three constraints: (1) target the most divergent “hot zones” between the two versions; (2) describe only observable inputs/outputs, leaving the exact code change to the agent; (3) run tests only on public interfaces (CLI, HTTP routes, library APIs). Each task was executed ten times on both sides using Claude Code (Sonnet 4.6), for a total of 660 runs.

Core Findings

Task success independent of cleanliness

Pass rates: 91.3 % on clean code vs 92.1 % on messy code.

Clean code reduces token consumption

Input tokens − 7.1 %

Output tokens − 8.5 %

Reasoning characters − 11.1 %

Since success rates are unchanged, the same work is done with fewer tokens.

File revisitation drops

Agent rereads previously modified files 34 % fewer times on clean code, ranging from − 7 % (CKAN) to − 69 % (Commons BCEL).

Refactoring location matters

Tasks were split into two tracks.

Cross‑module tasks (14 tasks): input tokens − 10.7 %, file revisits − 50.8 %.

Cognitive‑hotspot tasks (13 tasks): token footprint similar, but clean code opened more files with fewer line changes per file.

Case study 1 (Commons BCEL): replacing a massive switch with dozens of small helpers reduced input tokens by 35 %, file opens by 25 %, code changes by 31 %, and dialogue rounds by 32 %.

Case study 2 (Netflix Genie): refactoring that only redistributed logic increased input tokens by 8 % because the agent had to traverse more files.

Comment ablation

Annotations and suppression markers ( # noqa, // NOSONAR, # TODO) were equalized across pairs without changing logic. When original comment volume was similar, token usage changed negligibly. When comment disparity was large, removing the excess comments amplified the token‑saving advantage of clean code (up to − 18 % input tokens).

Limitations

Sample size: six repository pairs and 33 tasks.

Only Claude Code + Claude Sonnet 4.6 was evaluated; other models showed insufficient pass rates for comparable analysis.

High variance: individual runs varied by up to 2.5× in token cost; 16 of 27 tasks saved tokens with clean code, 11 saved more with messy code.

Implications

Code cleanliness does not affect whether an AI coding agent can complete a task, but it lowers token cost and reduces redundant file accesses. Token savings of ~7–8 % per task can accumulate across many agent invocations, and a 34 % reduction in file revisitation indicates more confident navigation in tidy codebases.

Reference: https://arxiv.org/html/2605.20049v1