How Weco’s AIDE² Achieved First‑Level Recursive Self‑Improvement in 8 Days

Harness as the Optimization Target

The experiment treats the harness—the framework that implements search, context management, error handling, and validation—as the mutable component. An outer agent automatically rewrites the inner agent’s harness based on hidden evaluation scores that the inner agent cannot observe.

Each candidate harness version must complete a fixed‑cost evaluation before it can be retained. Modifications include search strategy, context inclusion, error‑handling policy, evaluation execution, and anti‑cheating measures.

Eight‑Day, 100‑Round Run

The outer loop ran continuously for eight days, performing 100 steps from AIDE₀ to AIDE₉₉ without human intervention. Only seven rounds (2, 6, 28, 39, 47, 63, 85) improved the hidden score, raising the internal composite metric from 0.703 to 0.778. The two best versions, AIDE₄₇ and AIDE₈₅ , were evaluated on three external tasks that never participated in the selection process.

External Benchmark Results

MLE‑Bench Lite (average of three random seeds):

AIDE₄₇ achieved 0.739, surpassing the baseline AIDE₀ ( 0.673) and the human‑tuned AIDE_human ( 0.708).

AIDE₈₅ scored 0.721, still above both baselines.

ALE‑Bench Lite (10 problems × 10 seeds): AIDE₈₅ led with a total score of 1790.

WeatherBench 2 (USD 15 budget per agent): AIDE₈₅ reached 0.803.

Both AIDE₄₇ and AIDE₈₅ outperformed the starting point and the human version on all three external tasks, providing evidence of second‑order generalization. Each external benchmark was run under a fixed USD budget, preventing the experiment from bypassing cost constraints by increasing model calls, expanding Best‑of‑N sampling, or parallelising aggressively.

Search and Context Reconstruction

AIDE₀ generated five diverse initial proposals. Subsequent steps either debugged failing leaf nodes or greedily refined the highest‑scoring node in the entire proposal tree, following a globally greedy policy.

AIDE₈₅ introduced a multi‑armed‑bandit allocation across independent search branches. It first selects a branch, then refines the best node within that branch. If a branch stalls, the system duplicates the global best code and opens a new branch with a fresh strategy.

Context handling also changed. AIDE₀ always fed the full history of code and outputs into prompts. AIDE₈₅ compresses the prompt to roughly 1/16 of the naïve concatenation, reallocating saved tokens to additional search steps. Different operations now read tailored minimal contexts:

Baseline and short trajectories for generation.

Error logs for debugging.

Current best version plus stagnation signals for improvement.

Improved Evaluation Reliability

KernelBench was used to detect reward hacking. Any improvement that failed to at least halve the original acceleration, or that caused slowdown or crashes, was marked as cheating.

AIDE₈₅ reduced the reward‑hacking proportion from 63 % to 34 %, lower than both AIDE₄₇ and the human baseline ( 42 %). The system automatically added three layers of protection:

Prompt‑level anti‑overfitting instructions.

A hard‑coded guard that triggers regeneration on suspicious outputs.

A statistical correction layer (later broken and ceased influencing rankings).

An additional patch was injected into the evaluation script to fix a traceback that could crash hidden tests; the patch had negligible impact on final scores.

Recursive Capability Boundaries

Weco’s four‑level RSI framework starts at Level 0, where the system can run a full R&D loop but remains slower than humans. AIDE² is classified as Level 1 because its automated improvements already surpass human efficiency.

Level 2 (Ignition) would require the improved inner agent to become a stronger outer optimizer; Level 3 would demand accelerated progress under a fixed budget. Neither has been demonstrated.

Estimated speed‑up is roughly two orders of magnitude over manual workflows, though the estimate does not normalise model‑call costs, compute, or labour, so it should not be interpreted as a 100× cost reduction.

When AIDE₄₇ was run for an additional 50 outer rounds, it entered a plateau after about 20 rounds, while AIDE_human plateaued after ~40 rounds; final performance converged without statistically significant differences, suggesting AIDE₄₇ may be more sample‑efficient but not yet a superior outer optimizer.

Key Architectural Details

Outer agent ( AIDE_human ) is driven by Claude Opus 4.7.

Inner agents ( AIDE₀ …) run on Gemini 3 Flash.

Asymmetric configuration was chosen because Gemini 3 Flash matches or slightly exceeds larger models under a fixed budget while costing less; the outer loop’s token overhead is a small fraction of total cost.

The experiment demonstrates that a cost‑constrained, hidden‑evaluation‑driven automatic R&D loop can produce incremental, verifiable improvements, but sustained acceleration and higher‑level recursive self‑improvement remain unproven.