AntOmniEvo: Let AI Automatically Optimize Your AI Systems

Ant Group's Lingxi Large Model Application Platform team has open-sourced AntOmniEvo , a framework that automates the optimization of AI systems — turning the manual, expertise-heavy process of tuning prompts, rules, scripts, and configurations into an autonomous, 7×24 evolutionary loop.

Core Idea: You Provide Four Inputs; the Framework Does the Rest

Users supply:

Tunable artifacts — a real file directory mapping the mutable parts of the system (prompts, rules, scripts, configs).

Evaluation data — test cases.

Scoring criteria — how to grade outputs.

System definition — how the system runs (the System interface).

The framework then drives a coding agent to repeatedly modify files, run evaluations, and improve scores until a budget is exhausted. The deliverable is the optimized artifact directory — directly deployable, with every change diffable, reviewable, and rollbackable.

System Architecture: 7 Pluggable Components

The framework separates generic machinery from task-specific plugins:

Target side : System (how your system runs), Evaluator (how to score), DataInst (how to load eval data).

Optimizer side : Optimizer (main evolution loop), Proposer (coding agent that edits files), EvolutionAlgorithm (selection/mutation strategy).

Persistence : CandidateStore (stores all state; swappable for cloud).

Integrating a new task typically requires implementing only System and Evaluator; the rest have defaults.

Algorithm Layer: Two Key Designs

Mara Chain — Don't Discard Failures Too Early

Standard evolutionary approaches discard candidates that don't immediately beat the baseline, wasting budget on repeated diagnosis of the same errors — a persistent failure barrier . Mara Chain retains failing candidates along with their diagnostic records, giving them a limited correction window to iteratively fix issues. Each generation inherits four evidence assets from the chain:

Per-question score history

Ancestral diagnostic reports (what was wrong, which file to change, how)

Per-generation edit logs (file, reason, linked to score changes)

Raw execution traces (why scores changed)

This allows the next generation to continue from partial fixes, correct implementation bugs without abandoning a valid direction, or pivot after a path is proven dead. The chain depth is capped; only the best candidate that surpasses the chain's starting point proceeds to validation.

Case study (sanitize-git-repo) : Without Mara Chain, the system repeatedly proposed half-fixes that were discarded, never solving the task. With Mara Chain, failures were iteratively refined — from textual rules to executable scripts — until the task passed.

Pareto-Filtered Top-N Selection — Keep Specialists

Pure total-score ranking eliminates candidates that excel on a few hard questions but have mediocre averages. The framework first applies per-question Pareto filtering to preserve these specialists, then truncates by total score to control population size. Ablation shows disabling either mechanism increases rollouts to reach the same score by ~1.5× (Mara Chain) and ~2.3× (Pareto), and final scores fall short of 0.87. On TerminalBench, removing Mara Chain drops pass rate from 71.9% to 57.3% (-14.6 pp), with larger gains on harder tasks.

Engineering Foundations: Build Once, Reuse Everywhere

The framework provides production-grade infrastructure so each integration doesn't reinvent the wheel:

Five-dimensional budget control — iterations, rollouts, system runs, tokens, wall-clock time; soft-stop on any limit.

Cost monitoring — per-run and per-AI-action token accounting, real-time cumulative view.

Progress monitoring — per-iteration scores, candidate counts, budget consumption.

Checkpoint resume — all candidates, runs, edits persisted; survive machine crashes on 10+ hour jobs.

Multi-slot concurrency — multiple candidates evolve in parallel sharing a candidate pool.

Full audit trail — every edit forced into a changelog (file, reason, linked failure evidence); proposer agent's own traces also logged.

Validation-set hard isolation — validation data invisible to the proposer at runtime; training set decides edit retention, validation set decides ranking.

Lobster Gym visualization — renders the entire evolution as a "lobster gym": family trees, score trajectories, ancestor chains, per-question coverage grids. A real run showed 623 iterations, 34 selected candidates, 590 eliminated, 2 days 3 hours unattended.

Benchmark Results: Three Benchmarks, Three System Shapes, All First Place

All experiments freeze the base model (default GLM-5) and compare against specialized baselines under identical conditions.

AppWorld (Skill shape, 585 tasks)

Four metrics (Task/Scenario Completion Rates) all #1: 89.9 / 83.9 / 76.7 / 59.0.

vs. unoptimized baseline: up to +48.2 pp (Normal SGC).

vs. strongest prior (GEPA): up to +20.5% relative improvement.

Sample efficiency : Reach validation 0.8 with 3,280 rollouts — ~1/3 of GEPA's 9,514; wall-time 11.9h vs 35.8h.

Final ceiling: AntOmniEvo 0.870 vs GEPA 0.805. ACE and SkillOpt-Lite never reached 0.8 (capped at 0.770 / 0.665).

TerminalBench 2.1 (Harness shape, 89 tasks, 100h budget)

AntOmniEvo pass rate: 71.9% (+20.2 pp over unoptimized 51.7%).

Specialized harness optimizers did not beat baseline : AHE 51.7%, Meta-Harness 49.4%.

MuSiQue (Retrieval pipeline shape)

Optimizes pipeline configs, node scripts, prompts — not an agent.

Test nDCG@10: 0.301 → 0.405 (+0.104); Recall@10: 0.329 → 0.460 (+0.131).

Validation gains (+0.134 / +0.210) on disjoint sets confirm generalization, not overfitting.

Demonstrates the method works beyond agent systems.

Cross-Model Generalization

Optimized separately on GLM-5, DeepSeek-V4-Pro, Qwen3.5-397B; each shows average metric gains of +40.3, +32.7, +23.4 pp over its own unoptimized baseline — improvements are in the system, not tied to a specific model.

Limitations

On Text2SQL tasks, no performance improvement has been observed yet; causes and remedies are under investigation.

Summary

Comprehensive — optimizes any mutable artifact: agent skills, harnesses, retrieval pipelines; goal is optimize anything .

Efficient — SOTA on three benchmarks/shapes; ~1/3 rollouts and wall-time of strongest prior.

Usable — implement two interfaces; reuse Mara Chain, Pareto selection, budgeting, monitoring, resume, audit, validation isolation, and Lobster Gym. Define "good"; it works nights; you collect a deployable, diffable artifact directory.

Links: Framework code — https://github.com/ant-research/AntOmniEvo; Mara Chain paper — https://arxiv.org/abs/2609.35855.