OpenMLE: Tsinghua’s Self‑Evolving MLE System Pushes 35B Model Past GPT‑5.5

1. RSI as an Executable Testbed

Recursive Self‑Improvement (RSI) is described as the ultimate AI narrative where each system upgrade generates the next generation, forming a self‑reinforcing loop. The article argues that RSI has lacked a measurable, executable, and reproducible experimental arena.

Machine Learning Engineering (MLE) is presented as a concrete instance of AI‑for‑AI: an agent builds ML solutions for real tasks and iteratively refines them via execution feedback.

Figure 2 | Left: relationship between OpenMLE stack (environment/learning/search layers) and Frontis‑MA1; Right: mechanism hierarchy from evolution to RSI.

Evolution : mutation + selection, AI repeatedly modifies candidate solutions.

Meta‑Evolution : the evolutionary trajectory is recycled to train the “modifier” model, i.e., the improver itself is trained.

Self‑Evolution : experience feedback loop.

RSI : the ultimate goal where each upgraded system further improves its successor.

2. Super‑Evolution of a 35B Model

The benchmark MLE‑Bench Lite (OpenAI’s 22‑task split) runs each task for 12 hours on a single RTX 4090 (12 GB). This compute budget is smaller than most reported evaluations.

Figure 1 | MLE‑Bench Lite overall leaderboard: left shows Medal avg@3 for all model×harness combos; right shows Pareto panel of model size (log scale) vs score.

The Pareto plot reveals that the 35B Frontis‑MA1 alone dominates the small‑model frontier. Post‑training plus domain‑specific search compresses the ~80× parameter gap into only a 1.5 percentage‑point score gap.

Controlled comparison data (same harness) shows that post‑training adds +21.22 pp for the 35B model and +18.18 pp for a 30B cross‑base replica; switching to a stronger harness (Evo‑Max) adds another ~10 pp. The article emphasizes that training gains and search gains are composable, which is the key quantitative conclusion.

3. OpenMLE‑Gym: A 5,758‑Task Playground

The first bottleneck for training MLE agents is the environment: thousands of task packages with data, evaluators, and sandbox execution are needed, yet existing benchmarks provide only a few hundred (MLE‑Bench 75, MLE‑Smith 606).

Figure 3 | Task curation: left shows quality‑scale trade‑offs of three data sources; middle shows funnel filtering of competition tasks (11000 → 3972 → 2839 → 2240); right shows unified executable task package format (public input / private answer / executable metric.py).

OpenMLE‑Gym combines three complementary task sources:

Curated Anchors (156) : manually selected, highest quality but not scalable.

Kaggle Competitions (2,240) : human‑written problem statements with real leaderboards; pipeline filters ~11,000 competitions down to the top 20%.

Kaggle Datasets (3,362) : automatically induced tasks from datasets; largest scale but higher quality variance.

Figure 4 | Scale and composition: 5,758 tasks give an order‑of‑magnitude advantage over existing resources; modality breakdown: tables 44%, images 18%, time‑series 13%, multimodal 11%; task types: classification+regression 87%.

4. OpenMLE‑ERL: Turning Evolution Operators into Trainable Targets

Traditional approaches either train the entire trajectory (sparse supervision, tightly coupled to a specific controller) or only perform inference‑time search (model itself does not improve). OpenMLE‑ERL decouples evolution into four atomic operators that share the same interface for both training and inference.

Figure 5 | OpenMLE training and inference overview: four trainable atomic operators (Draft create / Improve refine / Debug fix / Crossover merge parents) serve both search‑space tree expansion during inference and SFT warm‑start / RL online optimization.

4.1 SFT Warm‑Start: 26,259 Verified Samples

SFT data are not distilled text trajectories but samples that have actually been executed in the sandbox and filtered by score.

The collection process is budget‑adaptive: sampling stops when a quota is reached or the execution budget is exhausted, allowing early termination of easy tasks and reserving compute for harder, sparse tasks.

Figure 7 | Learning from executed rollouts: left shows SFT dual‑path corpus construction; right shows RL parent‑node selection (reward + child‑reward variance + visit cooldown), Top‑1/Top‑K adaptive boundary reward normalization, and entropy advantage amplifying tail signals.

4.2 RL: Three Designs for MLE Scenarios

MLE‑specific RL differs from standard RL/VR because many programs produce no usable reward, scores come from heterogeneous metrics, and feedback latency ranges from minutes to hours. OpenMLE‑ERL addresses this with:

Adaptive Bounds : dynamically derive tighter reward bounds from each task’s historical frontier, remapping raw scores to [0, 1]; the bounds evolve together with the policy, preserving resolution in the current candidate region.

Entropic Advantage : applies exp(β·r) to amplify reward differences among top rollouts, replacing GRPO‑style group normalization. The effect is clear: the processed advantage of the best candidate rises from 1.58 to 6.39 (≈4×).

Asynchronous Rollout : generation‑execution groups start independently and enqueue results as soon as they finish, decoupling policy updates from the slowest tasks.

Figure 8 | Effect of tail‑reward shaping: (a) entropy weighting lifts processed advantage of the optimal candidate from 1.58 to 6.39 (4×); (b) combining adaptive bounds + entropy weighting yields stronger Group Best Reward trajectories, test medal rate 34.8±4.3 vs 24.2±5.7.

Figure 6 | Frontis‑MA1‑35B RL training curve: Validation Base Reward rises from ~0.15 to ~0.40; Validation Medal Count (176 tasks) rises from ~7 to ~25+.

5. OpenMLE‑Evo: Experience‑Driven Long‑Horizon Search

Trained operators must be combined by a search framework. OpenMLE‑Evo builds on the AIRA‑Evo population‑loop style but redesigns how execution evidence is used. Two coupled scientific questions are posed:

Which node should be expanded next beyond greedy score maximization?

How should memory be constructed so that selected operators receive actionable evidence instead of unbounded history?

Figure 9 | OpenMLE‑Evo search harness: left shows search tree expanded by Draft/Improve/Crossover; each evaluated node gets a structured experience card; right demonstrates an Improve step – update experience card → three‑factor scoring (quality 0.712 + progress +0.045 + novelty 0.707) → Softmax parent selection → memory injection from key ancestors and siblings.

The four core mechanisms are:

Structured experience accumulation.

Three‑factor parent‑node selection.

Memory synthesis triggered by operators.

Operator‑conditioned context.

6. Experiments: Training and Search Gains Stack

6.1 Harnesses Create Value

Swapping harnesses while keeping the model fixed shows OpenMLE‑Evo consistently outperforms generic programming agents such as Claude Code or Codex across four leading model families. For Frontis‑MA1‑35B, the score improves from 53.03% (original AIRA‑Evo) to 60.61%.

Figure 10‑11 | Harness comparison.

Under a unified OpenMLE‑Evo, Frontis‑MA1‑35B (60.61%) surpasses MiniMax M3, Doubao 2.1 Pro, DeepSeek‑V4 Pro, and is only behind Kimi K2.6 and GLM‑5.2, whose parameter counts are ten times larger.

Figure 10 | Model comparison under unified OpenMLE‑Evo harness: solid bars = standard Evo, hatched = Evo‑Max extra gain.

6.2 Long‑Term Self‑Improvement: Gains from Structural Re‑composition

Task‑level trajectories illustrate the mechanism. On the leaf‑classification task, competing models either plateau at low Human Rank or improve without reaching medal thresholds. Frontis‑MA1 first uses Debug to create feasible image and table branches, then Crossover to retain complementary evidence, and finally Improve to fuse the model.

Figure 13 | Leaf‑classification trajectory.

On the audio task mlsp‑2013‑birds, competing trajectories stall near the first feasible solution, whereas Frontis‑MA1 treats the repaired submission as a starting point and incrementally builds a dedicated audio branch. Memory’s role appears as selection rather than accumulation.

Figure 14 | mlsp‑2013‑birds trajectory: repair and re‑composition produce a silver‑level bird detector.

6.3 Solution Ceiling: More Gold Than Bronze

Figure 15 | Medal decomposition: post‑training and Evo‑Max push solutions beyond bronze into gold; Frontis‑MA1‑35B + Evo‑Max matches Kimi K3 gold rate, surpasses Claude Opus 4.8 + Claude Code and Gemini 3.5 Flash + Gemini CLI.

6.4 Efficiency Dissection: Fewer Tokens, Higher Yield

In a controlled experiment with identical checkpoints, seeds, 12‑hour budget, and 66 tasks per harness run, OpenMLE‑Evo outperforms the original AIRA‑Evo.

Figure 16 | Search efficiency comparison.

Two case studies illustrate the mechanism:

Figure 17 | Case 1: nomad2018 transparent conductor.

Figure 18 | Case 2 (fin whale detection): highest‑scoring parent A (AUC 0.99187) is not the most valuable; parent B ranks 6th but yields the biggest gain; adaptive weighting raises its selection probability from 10.47% to 17.09% (+63.2%), leading to held‑out AUC 0.99386 vs greedy 0.94852.

The article highlights that the highest‑scoring branch and the most valuable branch are often different; structural complementarity can be lost if selection relies solely on raw scores.

https://arxiv.org/pdf/2607.28568
https://github.com/FrontisAI/OpenRSI
Frontis-MA1: Training an AI4AI Model towards Recursive Self-Improvement in Machine Learning Engineering