Fei-Fei Li & Jeff Dean: Why Recursive Self-Improvement Isn't What Silicon Valley Thinks

Clarifying the Terminology: Self-Improvement vs. Recursive Self-Improvement

Fei-Fei Li opened by noting that the concept behind RSI has appeared under many names throughout ML history: incremental learning, continuous learning, meta-learning, lifelong learning, and neural architecture search (NAS). The current hype around "self-improvement" and RSI is just the latest label.

Jeff Dean provided a sharp criterion: what exactly is getting better? He illustrated with a medical imaging example. An automated pipeline that searches model architectures for MRI diagnosis will improve the MRI model's accuracy. But the underlying system that designs experiments, schedules runs, and evaluates results — the "base research system" — does not improve at all. That is ordinary self-improvement, not recursion.

True RSI, in Dean's view, requires that the improvement feeds back into the system that proposes improvements . Only when the base system becomes better at designing experiments and forming hypotheses does the process cross into genuine recursion. This distinction filters out ~90% of claims that conflate an application pipeline with an evolving research substrate.

Point Gains vs. Substrate Evolution

Li used a personal analogy: learning to make an omelet through trial and error is self-improvement on a single task. RSI would mean the brain's neural wiring fundamentally restructures — perhaps growing a sixth finger for flipping — so the learning machinery itself upgrades and transfers to harder tasks.

Dean agreed humans run both engines. We improve at specific skills (cooking, calculus) but also "learn how to learn" — a meta-learning ability that accelerates future acquisition across domains. Mapping to AI: the core challenge is whether a system, after solving many concrete problems, can extract a transferable "research methodology" that applies to unfamiliar fields. If not, it remains a skilled worker, not a recursive researcher.

Automation Cannot Bypass Physical Limits

Dean addressed the popular vision of infinite acceleration once AI runs its own experiments. He described the "2 AM case": today a researcher submits an experiment in the evening; it finishes at 2 AM but sits idle until the human returns at 8 AM. Automation eliminates that six-hour handoff gap, enabling 24/7 experiment loops — the thesis behind his new company Discovery Loop.

However, the compute time of each experiment cannot be wished away . Training requires matrix multiplications, backpropagation, data I/O — all bound by hardware throughput. Causality also forces sequential dependence: the next experiment's design depends on the previous result, so the wall-clock time of each run is a hard floor. Automation yields an "engineering dividend" by removing idle gaps, but it never overrides the physics of computation.

Dean noted Discovery Loop focuses on ML, engineering optimization, and chip design because their loops close entirely in the digital realm. Wet-lab domains (materials, chemistry, biology) are constrained by cell growth and reaction kinetics that no algorithm can fast-forward.

New Division of Labor: Machine Explores, Human Distills

Li, drawing on her physics background, asked whether massive automated experimentation might overturn the centuries-old theory/experiment duality. Great theoretical leaps (Newton, Einstein) came from insight, not brute-force trials. If AI science becomes pure high-throughput experimentation, does it collapse into crude empiricism?

Dean was optimistic: at sufficient scale, quantity births new theory . A system that has run massive tests across hundreds of domains holds cross-disciplinary empirical data and boundary conditions no human scholar could accumulate. From that corpus it can search for a "minimal description theory" that explains everything — potentially dissolving academic silos.

On mathematics, Li cited Fields medalist Terence Tao's reaction to an AI agent that proved a Navier-Stokes result via massive parallel search in Lean. The proof spanned hundreds of pages of formal verification code — correct but opaque. Mathematicians value not just truth but the new tools and insight generated en route.

Dean acknowledged the aesthetic gap but emphasized the strategic value: "It tells humans: this problem has a solution in the current logic." That certainty is a powerful motivator. Humans then chase the elegant, human-readable path. The partnership: machines do violent exploration; humans distill tools and understanding.

Reality Check: Current Speedups Are Assistive, Not Recursive

Industry data from METR shows Anthropic's model development velocity at 1.9× the prior trend. The report clarifies this comes from AI helping engineers write code and handle tests — human-AI collaboration, not autonomous recursion. This aligns with Li and Dean: today's gains are workflow and tooling improvements. The practical question is not about abstract RSI definitions but: is the system helping humans go faster, or has it genuinely learned how to do research?