Can Multi-Model Co-Evolution Shatter the Single-Model Ceiling? Squeeze Evolve Achieves Validator-Free SOTA Inference

Research Background

Large language models (LLMs) quickly hit a capability ceiling: increasing inference budget or generating more candidates merely repeats the same priors, failures, and blind spots, causing answer populations to converge and stagnate. Test‑time scaling (TTS) can deepen reasoning by iteratively generating and recombining candidates, but it traditionally relies on external validators, which are often unavailable or too costly in domains such as plasma simulation, wet‑lab experiments, and open‑ended mathematics.

Validator‑Free Evolution Concept

The authors propose Squeeze Evolve , a multi‑model evolutionary framework that coordinates models with complementary strengths, failure modes, and inference styles without any external validator. Diversity is preserved by allowing models to err in different ways, turning the ensemble into a capability amplifier rather than a mere cost‑saving trick.

Key Empirical Findings

Initialization dominates final accuracy: The quality of Loop 0 (the initial population) predicts final performance. On the AIME 2025 benchmark, swapping the roles of the initialization and recombination models drops accuracy by up to 23 percentage points, highlighting the need for a strong seed population.

Weak models become powerful aggregators when the candidate set is strong: Once a group contains a correct trajectory, even much smaller models can aggregate it to near‑100 % accuracy. Expensive models only retain an edge on the hardest, most uncertain groups; elsewhere, cheap models are sufficient and efficient.

Group confidence predicts where capability is needed: Confidence derived from token log‑probabilities (Group Confidence, GC) cleanly separates groups that contain correct solutions from those that do not. This zero‑cost signal works across model families and directly informs which groups should be handed to expensive models versus cheap ones.

Method Overview

Different models bring distinct priors, training data distributions, and failure patterns. By evolving them together, the system maintains a complementary lineage that a single model cannot sustain. For example, a reasoning‑focused model may excel at multi‑step logical chains but falter in spatial tasks, while an instruction‑fine‑tuned model contributes a different inductive bias, preserving solution paths that the reasoning model would prune.

Experimental Evaluation

The authors evaluated Squeeze Evolve on several benchmarks:

AIME 2025: Combining GPT‑OSS‑20B with GPT‑5 mini achieved 55 % lower cost while surpassing GPT‑5 mini alone (95.4 % vs 94.2 % accuracy).

MMMU‑Pro: A mix of Qwen3.5‑35B‑A3B and Kimi‑2.5‑Thinking outperformed the single Kimi‑2.5‑Thinking model at 43 % of the cost (79.1 % vs 78.6 %).

ARC‑AGI‑V2: Gemini 3 1 Pro reduced RSA‑style cost by 3.7× and lifted accuracy from 93.3 % to 97.5 %.

Circle‑Packing problem: An open‑source pair (GPT‑OSS 120B + 20B) matched the closed‑source AlphaEvolve baseline that uses Gemini‑2.0 Pro + Flash, despite lacking any validator.

Across eight benchmarks, Squeeze Evolve reduced cost by 1.4–3.3× and increased throughput by 4–10×.

Conclusion and Outlook

The central insight is that the ceiling of a single model does not bound a system of cooperating models. By unifying test‑time scaling methods within a shared evolutionary framework, the authors expose a design space where models are assigned roles based on marginal utility, yielding not just cheaper inference but genuinely stronger reasoning. This reframes test‑time scaling from “spend more on bigger models” to a “multi‑model system optimization” problem, where intelligent orchestration of existing models drives the next wave of inference breakthroughs.