How Anthropic's Claude Code Team Reviews Code: AI Agents First, Humans Only for Risk

Anthropic's engineering team published a detailed blog post explaining how they handle code review when AI writes approximately 90% of the code in the Claude Code repository. The core problem: AI generates thousands of lines of code in minutes, making human line-by-line review impossible. Their main repository saw merged changes grow from 2,500 in May to over 30,000 in September — a 12x increase in four months.

Shift Checks Left: Pre-PR Self-Review by AI

Before opening a PR, the author must run tests and builds locally using Claude, and for UI changes, provide screenshots comparing against design specs. Real terminal logs must be attached as evidence. Then a fresh-context Subagent runs internal "skills" for bug-finding and code simplification. Founder Cherny estimates this self-check step alone improves code quality 2-3x.

Living Review Guidelines: CLAUDE.md and REVIEW.md

To stop AI reviewers from nitpicking trivial style issues, the team updates CLAUDE.md (loaded on every review) and module-specific REVIEW.md files whenever a review pattern proves noisy. Good REVIEW.md files are short and concrete: the docs module says "it lies", the config module says "it misroutes". They limit nitpick counts, list skip directories, require file/line evidence for errors, and tell the agent to stay silent when nothing is wrong.

Multi-Agent Automated Review with Verification Voting

When a PR is opened, multiple agents hunt for bugs in parallel. Findings are not posted directly; a separate verification agent votes on each candidate. Only majority-approved issues appear on the PR. Pure style nits were removed after A/B testing showed they wasted time. Remaining comments are labeled: severe bug, minor cleanup, or pre-existing technical debt (so the PR author isn't blamed for legacy code). Since launching this system, the share of PRs receiving substantive comments rose from 16% to 54%.

Risk-Based Human Approval: Three Tiers

After all automated checks (bug review, /security-review, dozens of CI jobs) pass, human involvement depends solely on change risk — not on whether AI or human wrote the code.

Low risk: docs, copy changes, internal tools, or code behind feature flags. Trusted team members can merge with zero extra approvals once automation is green.

Medium risk: regular product logic, cache, auto-updater. Requires one human approval.

High risk: core billing, CI config, critical infrastructure. Requires two human approvals.

Accountability is explicit: the person who instructed the agent is the legal reviewer; agent approvals never count toward human quotas. Any new commit or manual conflict resolution invalidates prior approvals, forcing a full re-run.

Feature Flags and Fast Rollback

Merged code goes behind feature flags for internal dogfooding. Production issues are rolled back in seconds by flipping the flag — no redeploy needed. The team acknowledges a missing automatic circuit breaker: if rollback rates spike, the system should tighten approval gates automatically. Engineers insist this fuse be added before expanding the zero-human-review channel.

The 50-Round Agent Loop Incident and Four Convergence Rules

One night a CLI PR triggered a 50+ round loop between a coding agent and a review agent. Post-mortem via git blame revealed 59% of the lines the reviewer kept flagging already existed before the first review round — the reviewer was re-surfacing old debt, not new bugs.

Four hard rules were added to stop infinite loops:

Maximum 3 review rounds; if still failing, stop and let a human reassess the overall approach.

After the first rounds, only inspect new diff — never re-scan legacy code.

Group reported issues by bug type so the author can fix an entire class at once; never repeat the same comment.

Legacy debt goes to a nightly scheduled review agent, keeping it out of the daily release pipeline.

Results: single-pass PRs rose from 65% to 78%, PRs needing 5+ rounds dropped from 8.9% to 1.5%, average rounds fell to 1.32.

Continuous Improvement via an Eval Suite

Every production incident and rollback becomes a regression test case (eval). Engineers familiar with the failure confirm the bug location and add it to the suite. Any change to reviewer prompts, tools, or underlying models must pass all historical evals before deployment.

Human Focus: Five Things Machines Can't Do Well

With tedious nitpicking automated, engineers concentrate on:

Necessity: Cross-team PRs still get a human read to ask "do we really need this feature?"

High-risk and safety-critical code: Billing, auto-updater, CI config.

Architectural patterns: Code that will be copied across the codebase.

Evidence verification: Checking test outputs, command return values, screenshots — if the author can't articulate what automated checks might miss, a deep manual review follows.

Ultimate accountability: Humans own the production outcome.

The team emphasizes that no single engineer holds a full mental model of the entire codebase, and they don't need to. Before touching a module, they ask Claude for a fresh "explainer" slice of the current state, sometimes with a visual artifact. Open Slack collaboration and cheap AI rewrites mean discarding a PR early costs almost nothing.

"Slowing down does reduce error rates, but mostly because you ship less. We'd rather get faster at finding and fixing problems." — Internal Anthropic engineer

Reference:

https://claude.dev/blog/solving-code-review-at-anthropic/