How the Claude Code Team Does Code Review: AI-First, Tiered, and Human-in-the-Loop

Rethinking Code Review Purpose

When AI agents write code, change volume explodes. In Anthropic's monorepo, daily merges jumped from ~2,500 in May to over 30,000 in September. Claude Code has its own repo but faces similar throughput pressure. Traditional line-by-human review becomes a bottleneck. The team identifies two core responsibilities of code review: (1) catch and fix issues before release, and (2) build shared context across the team. This article focuses on the first.

The key insight: "Has a human read this code?" is a poor proxy metric. The real question is "What risk does this change pose, and which check — automated or human — best intercepts it?" This shifts review from a final gate to a continuous process, like spell-check while typing. Most changes are reviewed and fixed by Claude at least once before a human sees them, filtering routine errors and reserving human attention for higher-level scrutiny.

Scarce Resource: Human Time and Attention

Goal: keep human time on problems only humans can solve. Human cost appears at both ends: time spent reviewing, and time spent fixing incidents caused by bad code. Cutting review time but increasing incident time is a net loss. Target: most PRs across CLI, desktop, web, and mobile repos merge with at most one human in the loop (who can be the author), while revert rate and incident rate stay flat or drop. Metrics tracked: human approvals per PR (median and p95), human comment count, PR wait time for human action. Revert rate and incident rate are safety rails — if they rise, time has just been shifted elsewhere.

Lifecycle of a Change

A typical change flows through multiple review layers, from the engineer's laptop (or a Slack session with Claude) through a series of automated and human checks.

1. Validate Before Anyone Else Reviews

Most review happens before the PR is created. The team gives Claude runnable verification — test suites, builds, screenshot comparisons against designs — and demands executed results, not just "done" claims. Then a sub-agent reviews the diff in a fresh context. These habits come from their best-practice guides. Many engineers also run internal custom Skills pre-PR: one hunts bugs, one optimizes the diff, one verifies end-to-end behavior. UI changes are checked against design guidelines. Boris Cherny (Claude Code creator) says giving Claude a way to verify its own output is "the single most critical lever" for quality, estimating a 2-3x quality lift. Verification can be a bash command, full test suite, or browser/phone emulator test.

When automated reviewers repeatedly flag the same issue class, the team fixes it at the source: rewrite the review Skill, add a lint rule, or update guidance files: CLAUDE.md: tells Claude what the project is — stack, conventions, goals, non-goals. Loaded on every review; kept short in large repos with links to other docs. Reviewer flags violations and changes that make lines in this file stale. REVIEW.md: written by code owners to tell the reviewer what matters most in their area. The team audits whether the reviewer actually follows these instructions. Good guides are concise: define what "failure" looks like for that repo (e.g., hallucinated docs for a docs repo, misrouted traffic for a config repo), cap nitpicks, list skippable paths, require comments to cite specific files/lines, demand summaries start with issue counts, and explicitly say "no changes needed" when clean.

2. AI Code Review on Every PR

Before any human looks, every PR gets a Claude review. It reads the diff with full repo context and the PR's stated intent, asking: does this change solve the promised problem, fit the current system, or break other modules? For bug fixes, it digs for root cause. The public version — "Code Review for Claude Code" — is available on Team and Enterprise plans; locally, /code-review runs a lighter version. Internal rollout of the research preview lifted the share of internal PRs receiving substantive review comments from 16% to 54%. PRs over 1,000 lines almost always get findings; most small PRs pass clean. Admins can set per-repo review depth: Quick, Standard, or Deep. Deep is worth it when the cost of a missed bug far exceeds extra review time.

Findings are multi-agent verified before reaching humans. Multiple agents hunt bugs in parallel; a separate set of verifier agents votes on each comment. Only comments with sufficient agreement are published. Pure style, logging, or trivial cleanup comments are discarded. The team A/B tested these filters on their own PRs before adopting. Survacing comments appear as inline annotations, sorted by severity, tagged "Important", "Nit", or "Pre-existing" so pre-existing bugs aren't blamed on the current author. Every comment must be resolved: fix it or close with a rationale. Often agents handle fixes overnight, so authors open PRs next morning already past one auto-fix and re-review cycle.

3. Security Review with Blocking Authority

Security gets its own automated review pipeline. If either the bug review or security review has open comments, the PR cannot merge. You must fix or close with a terse reason (which also clears false positives). New commits that fix the issue auto-resolve the review item. The same mechanism can be built via the open-source security review GitHub Action or /security-review command; configuration is documented in the "Claude Code Automated Security Review Guide".

4. Automated Checks with Blocking Authority

Standard gates: lint, type-check, full-platform test suites, build. A typical CLI PR runs dozens of checks before human eyes. PR templates require "How was this tested?" and "How to revert?" — Claude can draft these, but the requester must sign off.

5. Human Approval by Risk Tier

Fixing docs carries different risk than touching the auto-updater, so review differs. A classifier reads the diff and assigns a risk tier; a second classifier double-checks whether a seemingly low-risk change still warrants a human. Certain critical paths always require human sign-off: anything code owners mark as sensitive, and all CI workflow configs.

Low : Docs, copy changes, internal-only features, incremental changes behind feature flags. Human approvals required: Zero — once automated and security checks pass.

Medium : Regular product features, changes to cache, auto-updater, system prompts. Human approvals required: 1 approval.

High : Highest-risk core paths, plus a few fixed special paths requiring two-person confirmation. Human approvals required: 2 approvals.

Only trusted authors (usually the owning team) get the lightweight path; others need a second approver. Agent approvals never count toward human quotas. Automation can green-light low-risk changes but cannot substitute for a required human approval slot. The person who initiates the change owns it — for agent-authored PRs, the guiding human engineer is listed as a required reviewer with the same accountability as if they wrote the code. An approval binds to a specific commit; if new commits land that the system can't vouch for (e.g., agent follow-up pushes or manual merge conflict resolutions), human approval resets.

6. Merge Queue, Feature Flags, and Staged Rollouts

Merge queue re-runs all checks against latest main, so two PRs that pass individually but conflict won't break main on merge. New features ship behind feature flags to Anthropic employees first; heavy internal dogfooding often catches regressions before users see them. Public release goes to latest (early-access channel) while stable lags behind.

7. Fast Detection, Instant Revert

Errors will slip through, so the last line is "detect fast, revert faster." Feature flags kill features instantly without a redeploy. Claude Code runs weekly on-call; every incident retrospective asks: which layer should have caught this? Detection currently relies on authors, on-call engineers, and revert/incident dashboards. No circuit-breaker yet (auto-tightening approvals when reverts/alerts spike); the team wants that guardrail before pushing more code to "zero-human merge." One engineer noted: slowing delivery reduces errors mainly because you ship less total code. They'd rather get faster at finding and fixing problems.

Making the Reviewer Smarter

The review process runs in a continuous improvement loop.

All production incidents become eval cases. The team collects historical PRs that caused incidents, with their fix-forward and revert commits, and has domain-familiar engineers confirm each bug's true nature and location. The reviewer runs against this eval set daily.

Noise measured by human action on comments. Catching more bugs is easy if you tolerate false positives, so they balance it. Every reviewer comment is categorized by how participants respond: fixed it, marked it wrong, explained it's intentional, or deemed it irrelevant. They also run the reviewer on known-clean PRs to measure how often it hallucinates harmful suggestions. Stated principle: prioritize catching real bugs over reducing review rounds.

Noise cost is higher when the author is an agent. Humans dismiss nits in seconds; agents obediently rewrite code, spawning new commits and new review rounds.

Review must converge. An agent author and agent reviewer can loop endlessly. One CLI PR went 50+ rounds overnight until a human intervened. git blame showed ~59% of repeatedly flagged code had existed since the reviewer's first pass — the reviewer wasn't finding new bugs, just digging up more old ones each round. The team had encouraged multi-round reviews because each pass caught something, but that's only justified for highest-risk changes. Now they track "rounds-to-pass" alongside recall, and reformed re-reviews:

Hard cap on rounds: after three rounds, reviewer asks author to step back and reconsider the implementation approach.

Subsequent reviews target only the diff: after initial rounds, re-reviews scan only new incremental changes, not the full diff.

Call out bug classes: require author to fix all instances of a pattern, not just the one line.

Ban re-raising already-published comments.

Additionally, the review agent nightly scans the existing codebase to surface latent bugs before they spread.

Core Domains Where Humans Must Stay in the Loop

Pulling a senior engineer out of deep flow to catch a trivial error costs far more than automated review. Anthropic's focus has shifted from "read every diff" to decisions machines struggle with:

Should we do this at all? Direction, product taste, scope. Even a team that claims to have solved the review bottleneck still mandates human review for external contributions so someone can ask "Do we really need this?"

High-risk sensitive code: billing, auto-updater, core CI workflows.

Architectural evolution: core patterns and designs that other modules will copy.

Verifying evidence: reading test logs, actual command outputs, screenshots, reviewer summaries. If you can't articulate what automated checks might miss, do a full human deep-dive — don't rubber-stamp.

Owning the outcome: every merged change has a human fully accountable, regardless of who or what generated the code.

Cross-Team Context Sharing

Review's second duty — syncing codebase understanding across the team — traditionally happens by reading teammates' diffs. If machines do most reading, how do people learn the codebase? No rigid process. No one holds a full mental model of Claude Code, and nobody needs to. When an engineer wants to understand a specific area, they ask Claude to generate a detailed walkthrough of that slice — minutes, not hours, and it reflects today's reality. Some go further: have Claude build a lightweight interactive artifact (Artifact) that visualizes the code and explains it in the engineer's preferred style. This beats grinding through boring diffs; a one-line copy change teaches nothing, but a deep briefing on the module you're about to touch pays off. Default open Slack channels also help — you see what others are building in real time.

"You Land It, You Own It"

If you merge a small/medium change and it later breaks, you're on the hook to debug and fix. Safeguards prevent this from becoming a disaster:

Changes are small — easy to review, debug, revert.

Everything is reversible — feature flags, instant revert, staged rollouts fix most mistakes in minutes.

Blast radius sets the bar — expensive critical code gets more people and stricter checks.

Regressions surface fast — telemetry and internal dogfooding catch anomalies immediately.

Blameless culture — you own the follow-up because you have the most context; retrospectives target system/mechanism flaws, not individuals.

Ownership is also shared. Most work happens in open channels; anyone can watch, discuss, or direct Claude to shore up weak spots. Constructive feedback often comes from non-owners. This transparency is the bedrock that lets them maintain quality without "line-by-line defense."

When We Decide to Scrap and Restart

Agents make starting a project or writing code extremely cheap, so much work shouldn't merge. A casual question in chat — "Why this approach?" — referencing an earlier decision, or suggesting a lighter path, often leads the author to close or completely reshape the PR. Because code is fast to write, closing a PR doesn't feel like sunk-cost loss. The key: raise strategic alternatives early in the discussion thread, not as blocking nits on a finished PR.

How to Adopt This in Your Team

Most pieces are available today.

Before Opening a PR

Give your agent runnable checks, listed in a concise CLAUDE.md, with hooks that auto-run formatters and linters.

Run /code-review and /security-review locally, or spin a fresh-context sub-agent review.

Add a REVIEW.md capturing recurring human review feedback.

In Every PR Pipeline

Require green CI; add "Testing", "Risk", and "How to revert" to your PR template.

Pilot an AI reviewer (e.g., Code Review for Claude Code or Claude Code GitHub Action) in one repo, tune REVIEW.md for weeks, then expand.

Enforce automated security review; require all review threads resolved before merge.

Match review depth to risk: Quick for routine, Deep for high-cost misses (Quick/Standard/Deep in Code Review).

Cap review rounds; after initial passes, re-reviews target only new diff.

At Approval Stage

Define risk tiers based on the code change itself , not the author; start strict, relax with data.

Keep a fast lane for trusted authors.

Use CODEOWNERS and branch protection to ensure auth, billing, CI/publish workflows, and install scripts always need human approval.

Bot approvals never count toward human slots; any new commit invalidates existing human approvals.

Merge and Beyond

Use merge queue, feature flags, internal dogfooding, staged rollouts, one-click fast revert.

Turn historical incidents and reverts into a standardized eval set; re-run full suite whenever reviewer prompts, tooling, or underlying model change.

Continuously measure human review time; use revert rate and incident rate as floor guardrails.

Future Evolution

The public Code Review launch stated: approving a PR remains a human decision; the product emits "Findings", not "Approvals." Anthropic's own repos go further: repo owners decide which low-risk changes can merge purely on automated green checks, and that boundary moves forward as trust grows. Every time automated review misses a defect, they trace back: what context was the model missing? Then they codify the fix into CLAUDE.md, add a detection Skill, or add an automated check. The north star stays fixed: less human effort per change, while keeping revert rate no higher than before.

If your team already lets agents open PRs, they'd love to hear: what checks did you automate? What still requires a human? What blind spots do you see in their current approach?