AutoSOTA Finds 105 New SOTA Models in One Week, Restoring AI Research’s Creative Core

Problem

Researchers spend large time on repeated experiments that yield ≈1% performance gains; incremental optimization consumes months of experiment iteration, parameter tuning, and engineering.

State‑of‑the‑Art (SOTA) benchmarks are the gold standard for measuring research value. Achieving a new SOTA often requires sustained effort. Example: the Transformer architecture’s performance on the GLUE benchmark rose from ~75% in 2017 to >90% after many variants and training strategies.

AutoSOTA

AutoSOTA is an end‑to‑end AI research automation system announced by Tsinghua University and Beijing Zhongguancun Institute. The preprint is available at https://arxiv.org/abs/2604.05550 and the project site is https://tsinghua-fib-lab.github.io/AutoSOTA/.

It extends AI agents beyond isolated code‑optimization to cover experiment preparation, execution, literature mining, idea generation, and design, aiming to transform “from existing SOTA to new SOTA, from current code repository to new repository”.

Architecture

AutoSOTA uses a multi‑agent collaboration framework that mirrors the division of labor in human algorithm research. Agents cooperate on design, execution, result analysis, and iterative refinement. A toolkit and a skill‑set module handle complex runtime conditions and high‑level tasks such as literature review and hypothesis generation, forming a closed loop between high‑level planning and low‑level execution.

Empirical Evaluation

In a one‑week trial, AutoSOTA started from top‑conference papers of the previous year and automatically discovered 105 model proposals with significant performance improvements. More than 60 % of the proposals introduced novel architectural designs, and the average performance gain was close to 10 % over the baseline.

The results show that AutoSOTA does not merely perform brute‑force hyperparameter search; it also generates structural innovations that improve performance, demonstrating that AI agents can explore new design spaces beyond existing pathways.

Implications

The system suggests a research collaboration mode where repetitive, long‑duration optimization is delegated to intelligent agents, allowing human scientists to focus on problem definition, direction setting, and creative mechanism design. In this view AutoSOTA functions as a “creativity amplifier” rather than a replacement for human originality.