The Last AI Built by Humans? A Five-Level Roadmap to Recursive Self-Improvement

Introduction: From Scaling Burden to Recursive Improvement

The paper The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement addresses the growing complexity of frontier model development. Researchers now face multi-dimensional scaling across parameters, context, data pipelines, training experiments, inference calls, evaluation, and human coordination. The bottleneck is no longer single training cost but the end-to-end improvement loop: humans must still decide what to improve, how to improve, what feedback validates improvement, and which failures to retain.

The authors define Recursive Self-Improvement (RSI) as an autonomous closed-loop capability: an AI system identifies limitations, proposes and executes improvements, uses feedback to select retainable changes, and lets those changes influence subsequent tasks or improvement processes. True RSI requires three conditions: improvements must close the loop, results must be durably inherited, and the improvement process itself must gradually be taken over by the AI.

Three Core Challenges: Avoiding Common Confusions

In-task vs. durable improvement: Many models reflect and retry within a single task, but if experience does not alter system state for future tasks, it remains a local reasoning trick.

Automation vs. autonomy: Automated pipelines execute human-preset operations (batch labeling, hyperparameter search, candidate code generation) but goals, strategies, evaluation criteria, and retention rules remain externally fixed.

Performance gain vs. recursiveness: An agent improving on one benchmark does not mean it improved the mechanism that produces future improvements. The key is whether inherited changes affect how next-generation improvements are proposed, evaluated, and solidified.

Background: Uneven Capability Progress and Paradigm Boundaries

The paper introduces the Headroom-Closed Index (HCI) to compare progress across capability domains relative to observable ceilings. Aggregating 393 model-benchmark observations across ten categories (general knowledge, graduate science, mathematics, academic breadth, multimodal, legal, cybersecurity, software engineering, tool use, search/terminal workflows), results show faster gains in general knowledge, parts of science, and multimodal, while software engineering, tool use, and search/terminal — domains closer to real workflows — retain large headroom. The authors argue this residual space is where RSI can matter: when tasks require multi-round experimentation, tool calls, environment feedback, and experience inheritance, single-model capability gains are insufficient.

RSI overlaps with continual learning, AutoML, and Agentic AI but draws stricter boundaries: continual learning retains knowledge from new data but does not necessarily let the system autonomously decide improvement strategies; AutoML searches models or configurations but with fixed search spaces, objectives, and selection rules; Agentic AI plans and uses tools but improvements often stay within the current task and do not durably alter future behavior.

Five-Level Autonomy Framework for RSI

The central contribution is a five-level framework classified by which responsibilities of the improvement process the AI internalizes.

Level B0: In-Task Improvement (Non-RSI Reference)

Models reflect, search, or make multiple attempts within a single task, but improvements are not persistently inherited.

Level L1: Improvement Execution Autonomy

Humans specify goals, methods, and success criteria; AI executes candidate updates. Examples: batch labeling by fixed rules, automatic data generation against given evaluations, prompt rewriting by fixed templates.

Level L2: Improvement Strategy Autonomy

Goals and evaluations remain externally fixed, but AI diagnoses weaknesses and chooses how to improve — deciding whether to modify prompts, tools, workflows, or model configurations based on evidence. This covers prompt search, agent harness search, tool-call strategy adjustment, training recipe search. The paper emphasizes that strategy autonomy alone is not strong RSI because objective functions, task boundaries, and promotion rules may still be externally fixed.

Level L3: Learning Signal / Experience Acquisition Autonomy

The system decides what experience the next round needs. It may generate new tasks, construct curricula, select hard examples exposing current weaknesses, synthesize data, or design more informative exploration environments. The paper notes that data pipeline automation does not equal L3; only when learner feedback shapes future experience acquisition (e.g., generating targeted tasks from failure patterns) does it approach experience autonomy. Cited systems: R-Zero, SIMA 2, formal proof task generation, visual-language problem generation.

Level L4: Deployment and Environment Adaptation Autonomy

The improvement loop moves into real or near-real deployment environments. The system observes task outcomes, user feedback, tool failures, environment changes, and long-term usage patterns, converting them into reusable state: textual experience memory, structured knowledge bases, programmatic skill libraries, executable tools, agent harness modifications, or preference model updates. The paper warns that L4 risks inheriting errors: without clear verification, rollback, and boundary control, systems may encode accidental experiences, faulty rules, or adversarially poisoned feedback.

Level L5: Recursive Inheritance Autonomy (Meta-Improvement)

The system improves the mechanisms that produce subsequent improvements: candidate generators, evaluators, research strategies, resource allocation policies, or self-modifying programs. The paper distinguishes structural L5 (AI-directed changes durably affect later improvement rounds) from effective L5 (such changes yield better successor systems under independent evaluation and comparable budgets). A system that can rewrite its own code does not guarantee its improvement mechanism actually improves.

RSI Across Application Domains

Scientific Discovery

Science RSI faces openness and verification cost. It must generate experiments, collect evidence, interpret failures, update research strategies, and transmit verifiable experience. Feedback is expensive, noisy, and hard to attribute, requiring strong experiment logging, hypothesis management, and uncertainty control. The future shape is not automated single discoveries but continuous improvement of the research process itself: better question posing, experiment design, resource allocation, failure accumulation, and inheritance of process knowledge.

Embodied Intelligence

Embodied agents naturally suit RSI because robot-environment interaction yields real experience. However, physical trial-and-error is costly, safety constraints are strong, and failures may stem from perception, control, environment, reward, or hardware. An embodied RSI system must not only improve policies but also decide which environments to explore, how to collect experience, how to reuse skills across tasks, and how to avoid cementing dangerous behaviors. Current systems show policy-level self-improvement or model-environment co-evolution but have not reached unconstrained recursive improvement. True embodied RSI requires closed loops of environment generation, policy learning, skill memory, verification, and safety boundaries.

Software Engineering

Software engineering is a prime RSI testbed because code, tests, logs, execution traces, and version history are naturally recordable, verifiable, and rollbackable. A coding agent can treat itself as an editable software project: detect failure patterns, modify prompts, tools, workflows, or code, and judge inheritance via tests and independent tasks. The paper discusses Darwin Gödel Machine, Gödel Agent, HyperAgents, AIDE2, pointing toward future systems that maintain traceable, rollbackable, verifiable improvement lineages, enabling cross-project reuse of successes and prevention of repeated failures.

Healthcare

Medical RSI has high potential but the strongest constraints: clinical environments forbid arbitrary trial-and-error, feedback loops are long, accountability is strict, data privacy and institutional differences are significant. Feasible early forms are governed knowledge, memory, and process improvements: systems update diagnostic rules, preference records, or workflows from case feedback, expert review, and long-term outcomes, but all durable updates must be traceable, verifiable, and revocable. This illustrates that RSI implementation is highly scenario-dependent: software engineering leans on automated tests, science on experimental validation, robotics on physical safety, healthcare on expert oversight and institutional review.

Industry Landscape and Preliminary Practices

Environment-Data-Model Co-Evolution (Theseus)

Theseus argues that environments, data, and models must co-evolve: reconstruct real workspaces into executable environments, discover capability gaps from the environment to generate training data, train task models, then let stronger models further evolve the environment. Early workspace experiments show clean workspaces yield 21.7 to 51.6 percentage point pass-rate improvements over noisy ones; with fixed model and harness, adding environment information like Collection Maps and Event Logs also yields significant gains. This demonstrates that agent performance depends not only on the model but on whether the environment enables reliable inheritance of knowledge, state, and feedback.

Enterprise Data Foundations and Experience Banks (Lark, Humanlaya, Tencent Hunyuan Hyra, Agent-Native Research Lab)

Lark builds a closed loop from data collection, organization, quality assessment, feedback construction, to agent improvement, turning real usage feedback into next-round system improvements.

Humanlaya focuses on delivery-driven data quality assurance, distilling customer feedback, internal reviews, and error types into quality system improvement inputs.

Tencent Hunyuan Hyra uses context agents, parallel proposals, sandbox execution, evaluation, and an experience bank to retain solutions, logs, failure cases, and reusable knowledge for subsequent exploration.

Agent-Native Research Lab emphasizes verifiable research infrastructure to prevent evidence, experiments, and failure paths from being lost in paper-style summaries.

These cases show industrial RSI is typically constrained closed loops: humans define goals and boundaries, systems automatically collect experience, propose improvements, execute verification, and deposit reusable assets. Their value lies in grounding RSI from abstract vision to concrete engineering problems: how data continuously enters the system, how experience is structured, how updates are verified, how errors are rolled back.

Challenges and Future Directions

Evaluation and Attribution

A core difficulty is proving improvements stem from inherited system changes rather than task variance, evaluation leakage, resource increases, or external human intervention. The paper recommends tracking cross-round trends, retained states, independent transfer tasks, rollback controls, resource consumption, and failure cases — not just final scores.

Safe Inheritance

Durable inheritance is both necessary for RSI and a risk source. Systems that retain successes may also retain erroneous rules, biases, reward-hacking strategies, or adversarially poisoned memories. Future RSI systems need explicit change provenance, verification evidence, applicability scopes, revocation mechanisms, and protected evaluation sets.

Openness and Boundary Control

Approaching L5, systems may modify candidate generation, evaluation, and resource allocation mechanisms, increasing innovation potential but also unpredictability. The paper stresses that even at L5, mission objectives, safety boundaries, protected evaluations, and final acceptance authority should remain external constraints. Genuine RSI does not mean unbounded self-release but a verifiable, governable, sustainable improvement loop.

Conclusion

The paper provides a clear roadmap for RSI: from in-task reflection to execution improvement, from strategy selection to experience acquisition, from deployment adaptation to meta-level recursive inheritance. Its key contribution is decomposing "AI self-improvement" from a vague slogan into observable system structure: where the loop closes, what state is inherited, which decisions the AI takes over, and which controls remain external.

For current AI development, RSI matters not only for making models stronger but for making the improvement process itself more efficient, accumulable, and verifiable. It connects automated research, software engineering agents, enterprise knowledge systems, robot learning, and medical workflows, while reminding us that stronger recursive improvement demands rigorous evaluation, attribution, and governance.

If foundation models answer "how much can one model learn?", RSI asks "can a system continuously learn how to make itself better?" This is the unavoidable question as AI systems evolve from tools into long-term self-adaptive infrastructure.

Appendix: Literature Landscape

The appendix surveys 491 related works, categorized by autonomy level and improvement target. L1 and L2 dominate at 43.8% and 31.6% respectively; L3 accounts for 13.0%; L4 and L5 remain scarce at 5.7% and 5.9%, indicating current research concentrates on execution and strategy selection, with deployment adaptation and meta-improvement still early. Improvement targets span prompts/context, memory/knowledge, workflow control, tool skills, model weights, training optimization, evaluation/feedback systems, data environments, external artifacts, and full-system co-evolution. Future high-impact RSI systems will likely co-evolve multiple components within controlled boundaries rather than modifying a single component.

Code example

来源：专知
本文
约5000字
，建议阅读
10
分钟
这篇论文给 RSI 提供了一套清晰路线图：从任务内反思到执行改进，从策略选择到经验获取，从部署适应到元层级递归继承。