Claude Writes 80% of Anthropic's Code—Why That's Not True RSI

One: Defining RSI in Plain Terms

Ordinary automation runs a process faster. An agent receives a task, calls tools, modifies files, runs tests, and returns results. The system then stops. RSI adds a critical step: persisting experience so it changes the next task's starting point.

Goal & Constraints
 |
 v
Agent selects and executes a change
 |
 v
External environment returns results and failure reasons
 |
 v
Independent verifier judges if improvement is real
 |
 v
Candidate experience enters regression tests and version store
 |
 v
Next task reads it; on failure, roll back

A model saying "file generated" is just a claim. The file must exist, pass format checks, and be readable by downstream programs before it becomes evidence for the verifier.

A reusable cross-task experience must record at least five things: applicable inputs and environment, what was done, evidence of validity, boundary conditions where it does not apply, and the rollback version. It can be code, a skill, a tool configuration, or an evaluation rule—but not merely "last time this worked."

Walking Through a Training Optimization Task

Assume a concrete task: speed up a small model's training script without losing accuracy or exceeding memory budget. The input is an executable task contract: repository, fixed data slice, random seed, correctness tests, baseline runtime, and acceptance criteria (e.g., train 100 steps, validation metric not below baseline, per-step time reduced by at least 10%).

The agent works in this order:

Fix baseline : record Python version, CUDA, GPU model, batch size, current runtime.

Submit small change : e.g., adjust data prefetch or move a tensor copy to an async stream; keep diff, do not overwrite baseline.

Validate correctness first : run fixed-step micro-benchmark; discard diff if tests fail or speedup < 10%.

Re-test under changed conditions : swap data slice and random seed to confirm gain is not due to cache state or lucky input.

The verifier now holds test logs, timings, peak memory, and commit hashes—not the agent's "should be faster." Only when all evidence meets thresholds does the candidate enter an isolated branch.

A durable experience is not recorded as "async copy works." Below is a schematic record format (not actual Anthropic or RSIAgent data):

Applicable conditions: fixed-length batches, GPU free memory > 2 GB, CUDA 12.x
Change: commit 8f31c2, move batch copy into dedicated CUDA stream
Evidence: 100-step time 42.1s → 36.7s; validation metric diff < 0.1%; peak memory +0.4 GB
Not applicable: variable-length sequences with low free memory trigger OOM
Rollback: restore baseline commit 7b90aa

On a similar script next time, the runtime retrieves this record and replays in the new environment; if replay fails, the experience stays in the candidate zone and does not auto-apply because "it worked before." This process changes the next task's starting point and check order—qualifying as a bounded self-improvement.

Figure: Example task acceptance order. Red branches indicate candidate did not meet threshold, requiring revision or replanning.

Two: How Far Anthropic Has Come

Anthropic's article "When AI builds itself" splits R&D into engineering (writing code, building infrastructure, monitoring training) and research (choosing experiments, interpreting results, deciding what to try next).

Claude progresses rapidly in engineering. As of May 2026, Claude authored over 80% of Anthropic's merged code; in Q2 2026, a typical engineer's daily merged code volume was ~8× that of 2024. Code lines measure quantity, not quality. Engineers still set goals, review, and merge.

The real gap is research. For experiments with predefined goals and success criteria, Claude can modify code, run, time, compare, and iterate. But on "what to research next quarter," the system lacks stable judgment.

Anthropic also gives a verification-side example: if every code change passes automated Claude review, about one-third of past production defects could have been caught pre-deployment. As generation speed rises, the review chain must also become systemic.

Three: AL4 Describes Execution Authority, Not RSI

Discussions of Anthropic's R&D automation often cite Epoch AI's Automation Levels (AL), which describe how much operational responsibility humans retain in a task:

AL3 : AI completes large chunks under close guidance, stops on problems. Humans provide direction, supplement decisions, take over exceptions.

AL4 : AI executes end-to-end given high-level goal. Humans review results, decide whether to deploy.

AL5 : AI self-monitors, fixes, and deploys. No R&D subset has reached this.

AL4 answers "who finishes this task." RSI asks "does this execution change the system's capability for the next task." Conflating execution automation with completed self-improvement is a common error.

Theseus Labs uses L1–L5 to describe RSI autonomy scope: from executing human-written improvement processes, to selecting strategies based on evaluation, to deciding what experience and feedback are needed, to modifying and inheriting the "how to improve" mechanism itself. This grading is not an industry standard but helps locate public cases.

On this scale, current public cases mostly sit between "task executes automatically" and "experience has limited reuse," far from a system autonomously designing and developing its own successor model.

Four: How a Change Persists

RSI's difficulty is not adding another agent but separating responsibilities. At least four positions are needed:

Goal definition : inputs, outputs, permissions, budget, acceptance criteria fixed upfront; actions needing human approval also fixed.

Execution environment : resettable, replayable; tool calls, state changes, test results, resource consumption all auditable.

Verifier : reads only external evidence, judges if result meets goal, returns supporting evidence.

Experience release : candidate experience goes to isolation zone; after replay and regression tests it freezes; version and rollback pointer retained after release.

Engineering-wise, this can be a state machine:

draft : Write task contract, prepare baseline. Exit when inputs and acceptance criteria complete. Required evidence: task version, permissions, budget.

running : Agent modifies code, calls tools. Exit when tests complete or timeout. Required evidence: tool events, environment snapshots, resource usage.

candidate : Save candidate diff and experience. Exit when replay and regression pass. Required evidence: commit hash, test logs, comparison results.

frozen : Available for formal evaluation reads. Exit when failure detected or new version released. Required evidence: memory version, applicability conditions, rollback point.

rolled_back : Stop using this experience. Exit when fix and re-enter candidate zone. Required evidence: trigger cause, affected tasks, recovery version.

Every state transition writes to an append-only log so the runtime can answer "who published this experience in what environment." With only a final score table, the failure step and responsible change are usually lost.

Figure: RSI runtime separates candidate changes, external evidence, and frozen memory; verification failure returns to candidate zone or baseline.

Verification signals can be roughly graded by evidence strength:

Formal checks: compiler, type system, mathematical proofs.

Execution feedback: file state, interface returns, environment state, performance counters.

Learned scores from another model or classifier.

Model self-report: "looks done."

Earlier layers provide harder evidence but narrower coverage; later layers are cheaper but easier to fool with surface correctness and reward hacking. FunSearch and AlphaEvolve are relatively stable because candidate programs are executable, results computable, and the verifier need not understand full intent.

Open-ended research lacks such clean acceptance lines. One agent writes a proposal, another scores it—this does not automatically create independent verification; if both share the same blind spots, the system merely copies confidence.

Five: RSIAgent—Practice First, Then Pass Experience to Next Round

RSIAgent implements this chain as a multi-agent framework that does not update model parameters. It first explores in a new environment, building reusable memory; formal evaluation begins only after memory is frozen—no rule changes while answering.

Three roles each do one thing:

Curriculum Agent decides what to practice next based on existing memory and exposed problems.

Actor Agent calls code and tools to complete practice tasks in the environment.

Verifier Agent looks only at UI state, file results, and environment feedback to judge task completion.

The verifier cannot see the actor's private reasoning and memory. It may not always be correct, but it at least reduces correlated errors from "the test-taker also grading themselves."

Exploration proceeds broad-then-deep. BRS (Broad Exploration) tries different directions in parallel to learn environment, tools, and common workflows. DRS (Deep Exploration) pushes further along failures, knowledge gaps, and boundary conditions. After exploration, memory is frozen; formal evaluation then compares "with reused experience" vs. baseline.

The REAPER audio workstation example is concrete. The system first learned to select clips, concatenate, measure intervals, and export. Later it discovered resample settings affect results, and finally wrote the verified RENDER_RESAMPLE 0 0 0 command with its applicability conditions back to memory. What persisted was not "I did audio export" but a re-executable, re-verifiable experience.

The paper reports on 82 offline OSWorld 2.0 tasks: partial score rose from 71.97 to 78.98, full pass rate from 37.80% to 42.68%. On 67 Agents' Last Exam Near-term tasks: partial score from 83.75 to 84.82, full pass rate from 49.25% to 50.75%. These numbers show memory reuse helps, not that every task gained equally. The paper notes tasks without full RSI results used baseline scores.

Thus the accurate statement: model parameters did not update, but persistent experience changed subsequent execution. This is a bounded RSI implementation .

Six: GLM Case—Feedback Must Trace to Call Chain

Z.ai's GLM Infra Agent case is narrower but clearly shows "change, feedback, fix, re-test."

A GLM-5.3-driven agent optimized the inference system serving GLM-5.3-Flash. From first successful run to production-ready took under two weeks; end-to-end throughput reached ~3× initial baseline; Tang Jie's X post gave a specific 3.2× figure.

Problems mirrored real infrastructure work: limited GPU memory and interconnect bandwidth, input contexts up to 1M tokens, incomplete operators and documentation. The team added local correctness tests, execution traces, micro-benchmarks, and end-to-end measurements—each feedback close to the problem, cost-controlled, and verifiable.

One debugging session traced along timeline, thread state, and call chain to the Python/C++ boundary: the intra-node path did not release the GIL, so the thread responsible for submitting transfer tasks could not acquire the lock, preventing compute-transfer overlap. The fix was small: let the low-level call release the GIL during execution. After re-test, KV transfer overhead dropped below 1%.

Such results come not from a single "overall score" filter but from layered feedback pinpointing the exact code and call chain. Without load, control group, timeline, and local verification, an agent's suggestion struggles to turn from guess into engineering change.

Z.ai also explicitly states this is not full RSI. Humans still set goals, build feedback environments, and review high-risk changes. The case demonstrates the most critical RSI segment: experience must pass verifiable feedback to enter the next round.

Seven: Harness, Memory, Skill—Each Manages Its Own Layer

As models accumulate experience, the runtime must track state more granularly. An agent simultaneously handles at least four state layers:

Current context : this task's instructions, retrieved references, tool results.

Persistent memory : verified steps, constraints, and failure experiences ready for cross-task reuse.

Runtime state : task progress, tried versions, next handoff.

External authoritative state : code repos, production services, business databases, real delivery outcomes.

These four layers cannot impersonate each other. Context serves current reasoning; persistent memory handles cross-task inheritance; runtime state enables recovery and debugging; external systems provide ground truth. Skills describe process but cannot replace permissions; agents can report results but cannot alone constitute completion evidence.

Andrej Karpathy calls this context engineering: task instructions, examples, retrieved references, tools, state, history, and compression all selected for the next reasoning step. In RSI, we must further ask: which information belongs only to this exploration, which deserves long-term memory, and who has authority to publish that write.

Once experience enters long-term state, it should have provenance, version, applicability conditions, verification results, and rollback pointers—just like code and config. It is not a forever-valid prompt.

As agent count grows, supervision must split into online and offline chains: online monitoring intercepts irreversible actions (data deletion, model weight extraction, high-risk permission changes); offline audit reviews logs for rule bypass, test-report inconsistency, and creeping anomalous behavior. Online answers "what must not happen now"; offline answers "has it become easier to happen recently."

Eight: If I Were Building RSI Today, I'd Start Here

I would first pick a task with clear acceptance criteria—like the training script optimization above—not start with "let the model improve itself."

First, write the task contract: inputs, outputs, permissions, budget, stop conditions, acceptance criteria all fixed. The task must answer "which commit changed, how many steps ran, what hardware, result compared to what." Then build a resettable, replayable environment that saves tool calls, environment snapshots, test results, and resource consumption.

Separate exploration from formal evaluation. Exploration can broad-search and deep-search, allowing the system to discover failure paths. Formal evaluation reads only frozen experience, avoiding rule changes while scoring. A version boundary must exist between the two phases: which memory batch participated in evaluation, which was generated during evaluation.

The verifier reads only external evidence. Candidate memory undergoes replay, regression tests, and necessary human spot-checks before release. After release, retain online interception, offline audit, and rollback pointers. For performance tasks, at least put correctness, latency, resource peaks, and results across different inputs in the same acceptance record—so "faster" doesn't hide accuracy or stability regressions.

These steps don't make the model immediately write the next model, but they determine whether a change persists, whether experience transfers, and whether errors can be retracted. RSI's engineering threshold often lies in verifiers, state, and version management.

Closing Thoughts

Anthropic's public data shows Claude has deeply entered AI R&D's execution layer: code can be mass-produced by the model, experiments can be continuously driven by the model, and engineers shift from typing every line to setting goals, granting permissions, and reviewing.

RSI must go one step further. A successful debug, a practice run, a verification—must be organized into a record callable by the next task; when the record expires, the system must revert to the last reliable version.

So when I see "model wrote X% of code," I care about four things: did this change leave a cross-task artifact, what external evidence supports it, does it work under changed inputs and environments, and can it be retracted when it fails.

Confirmed RSI today is still far from "AI builds its own successor." The confirmed change is: systems have started passing what they did, what they verified, and where they failed to the next task for continued use.

References

Anthropic Institute, When AI builds itself (https://www.anthropic.com/institute/recursive-self-improvement)

Yi Duan et al., Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops (https://arxiv.org/abs/2607.07663)

Sibo Zhu et al., RSIAgent: Autonomous Exploration for Recursive Self-improvement in New Environments (https://arxiv.org/abs/2609.15364)

Yi Duan et al., The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement (https://arxiv.org/abs/2609.11873)

Z.ai, GLM Built Its Own Inference Infrastructure (https://z.ai/blog/glm-built-its-inference-infrastructure)

Z.ai, Public notes on GLM Infra Agent (https://x.com/Zai_org/status/2100481236364079277)

Tang Jie, Public notes on KV Transfer & GIL debugging (https://x.com/jietang/status/2100482019088060470)

Andrej Karpathy, Context engineering (https://x.com/karpathy/status/1937902205765607626)

Aaron Levie, Agent amplifies expertise & architectural trade-offs (https://x.com/levie/status/2006521312693637597)