Astar: Alibaba's LLM Guides AI System Evolution by Learning from Historical Commits

Introduction

Alibaba and Zhejiang University have proposed Astar , a specialized large language model designed to guide the evolution of AI systems by learning from their own historical version records — code commits, loss curves, and business metric changes. Unlike general-purpose LLMs that offer generic advice, Astar internalizes the real-world “evolutionary history” of a specific AI system to adaptively explore future improvement directions.

The Core Problem: AI Iteration Relies on Scarce Human Intuition

Industrial AI model iteration differs fundamentally from ordinary software engineering. A single change requires expensive retraining and offline evaluation, often taking hours to days. Teams cannot brute-force all ideas; they must rely on senior algorithm experts’ intuition to prioritize which direction to allocate compute. This expertise bottleneck limits iteration speed.

General LLMs (e.g., GPT-5.5, Claude-4.8-Opus) possess broad knowledge from public papers and open-source code, but they struggle to translate that into actionable, system-specific improvements. Engineers who follow such generic suggestions frequently see metrics stagnate or regress after days of training, wasting significant resources.

Astar’s Insight: Mine the System’s Own Evolutionary History

Every Git commit records what changed between model versions and the resulting loss curves and business metric movements. This constitutes a ready-made, business-feedback-rich “pitfall avoidance guide.” Astar’s team decided to turn these scattered historical records into training data for a model that learns how to evolve this specific system .

Four Challenges and Astar’s Solutions

Challenge C1: Data Too Sparse

Adjacent commits provide very few supervised samples. Astar adopts a pairwise augmentation strategy: pair any two historical experiments (e.g., version 1 vs. version 5), compare their full loss curves, and label the lower-loss version as better. This instantly expands sparse records into dense training samples, enabling the model to learn both short-term micro-adjustments and long-term leapfrog upgrades.

Challenge C2: Repository Full of Noise

Many commits are merely logging additions, file moves, or dead-code cleanup. Astar applies a two-stage filter:

Execution-logic filtering : using call-graph reachability analysis and abstract syntax trees (AST) to strip logging, dead code, and formatting changes.

Evolution-intent filtering : an LLM semantically analyzes the remaining clean code; changes that cannot be classified as a clear “optimization intent” are discarded. Only modifications that genuinely affect model performance remain.

Challenge C3: Search Space Too Large

Improvement directions are vast: architecture, loss function, optimizer, hyperparameters, etc. Astar uses hierarchical hints — three-level labels (primary direction → sub-module → concrete action) — during training. At inference, the model first decides the high-level strategy (e.g., “optimize optimizer”), then the sub-module (e.g., “Muon optimizer / gradient orthogonalization”), then the exact action (e.g., “gradient denoising”). This structured generation constrains the near-infinite search space into a tractable, stepwise process.

Challenge C4: Validation Too Expensive

Reinforcement learning requires massive trial-and-error feedback, but real experiments take days. Astar trains a reward model on historical positive/negative samples. Given a generated evolution proposal, the reward model predicts its probability of lowering loss in one second . This “proxy expert” enables lightning-fast initial screening of thousands of ideas and provides low-cost feedback for Astar’s own RL post-training, allowing it to explore directions never tried by humans.

Training Pipeline

Astar undergoes a standard three-stage training on the curated evolution corpus with the reward model: mid-training → supervised fine-tuning (SFT) → reinforcement learning (RL) .

Experimental Results

On a recommendation-system optimization task:

The smallest 0.6B Astar achieves a single-generation success rate (S@1) of 54.35% .

This surpasses the strongest general model GPT-5.5 (30.71%) and even human senior algorithm experts (32.29%) .

The 8B version pushes S@1 to 67.86% .

These results demonstrate that in vertical engineering decision-making, feeding a model high-quality, system-feedback-rich “domain evolution history” is as critical as scaling generic model parameters.

Efficiency Gains

Traditionally, a human expert might validate a handful to a dozen directions per week. With Astar’s second-level screening plus automated execution, the per-idea trial cost drops from hours/days to minutes — a 10–100× increase in iteration throughput .

Real-World Deployment on Lazada Core Ad Recommendation

Astar was given full control of Alibaba Lazada’s core advertising recommendation system for 20 consecutive fully automated iterations without human intervention. Results:

Offline HitRate@200 increased by 23.6% .

Online A/B test: GMV +4.86% , ad revenue +1.82% , with significant lifts in click-through rate and order volume.

Achieving such gains on a mature, heavily optimized, large-scale e-commerce ad system underscores the practical value of AI-driven evolution.

Case Study: Spectral Denoising for Muon Optimizer

When the team tried introducing the Muon optimizer, its orthogonalization property forcibly equalizes gradient direction weights. On clean data this helps, but in noisy recommendation scenarios it amplifies noise. Generic suggestions were “tune learning rate” or “revert to Adam.” Astar proposed a mathematically grounded fix: apply spectral denoising before orthogonalization . Using the Marchenko–Pastur law from random matrix theory, it computes a deterministic boundary: signals above the boundary are kept; those falling in the noise bulk are suppressed. This preserves Muon’s advantages while eliminating noise amplification, with zero extra training parameters. The fix exemplifies Astar’s deep understanding of the system’s data distribution and algorithmic mathematics.

Closed-Loop Evolution and Future Outlook

Astar operates in a closed loop: Astar proposes direction → Code Agent writes code & runs experiment → verification results feed back → new data retrains Astar . This makes Astar a “living” model whose guidance capability grows with the system.

The work fills a critical gap in automated AI evolution, making “optimization decisions” trainable and transferable for the first time. It does not replace algorithm engineers; rather, it heralds a new division of labor: humans design Meta-AI to guide AI . Engineers are freed from tedious trial-and-error to focus on frontier architecture breakthroughs, new business paradigms, and designing smarter “evolution flywheels.” The journey toward fully automated AI evolution has just begun, and human wisdom remains the indispensable navigator.