35B BigBang‑V1 Beats Trillion‑Parameter Models by Self‑Generating and Evolving Training Tasks

Recent observations by Ilya Sutskever suggest that simply enlarging pre‑training datasets will soon hit diminishing returns, prompting the AI community to revisit the research phase and seek new training mechanisms. The article frames this as a problem: as models grow stronger, human experts can no longer supply sufficiently novel and verifiable tasks.

BigBang‑V1 Concept

The Endless Frontier team at Shanghai Jiao Tong University proposes BigBang‑V1, a 35B‑parameter model built on Qwen3.6‑35B‑A3B, that addresses the problem by letting AI generate, filter, and evolve its own training tasks. The core idea is to replace the traditional human‑centric pipeline (human defines task → human provides answer → model learns) with an automated loop where AI produces both questions and answers and continuously refines the data‑generation program.

Two‑Level Loop Architecture

Inner Loop : Consists of a Generator Agent and a Critic Agent . The Generator continuously creates candidate tasks, executes them via a code‑agent, and records modifications, results, and failure reasons. The Critic performs a two‑stage review—first checking basic constraints (trace completeness, tool calls, data format) and then higher‑order quality (evidence for conclusions, consistency, verifiability). Tasks that pass become training samples; rejected ones are edited or discarded. The Critic’s feedback also shapes the Generator’s next‑round strategy.

Outer Loop (Meta‑Critic) : Evaluates the real training value of data batches by training the model on each batch and measuring performance on held‑out scientific tasks. If a batch receives a high Critic score but yields little model improvement, both the Critic’s criteria and the Generator’s synthesis strategy are adjusted. This outer loop calibrates the inner loop and drives a closed‑feedback flywheel.

Experimental Results

BigBang‑V1 was fine‑tuned with roughly 10,000 AI‑generated samples. On eleven benchmark tasks, it outperformed the base model on ten and matched or exceeded larger frontier models on several high‑difficulty tasks. Specific scores include:

FrontierScience Research: 46.2 (vs. DeepSeek V4 Pro Preview 40.7)

Humanity’s Last Exam: 50.3 (vs. 48.2)

BioMysteryBench‑HD and PaperBench (Code‑Dev): higher than competitors

BrowseComp (long‑range retrieval): 76.5

SWE‑Bench Pro (software engineering): 54.2

These gains appear without increasing the model’s parameter count, indicating that changing the data‑production pipeline can substantially expand capability.

Case Studies

Virus Identification : On BioMysteryBench, the task required classifying RNA viruses from FASTQ files of organoid samples. BigBang‑V1 consistently converged on the correct Norovirus GII.4 answer across three runs, whereas DeepSeek V4 Flash produced three different virus categories.

Elliptic Integral : The model had to analytically compute a complex elliptic integral without resorting to numerical approximation. BigBang‑V1 retrieved a public proof outline, verified each transformation (Landen, boundary conditions, parameter changes), and produced a closed‑form result using only the Gamma function, documenting a full derivation chain. DeepSeek V4 Pro also arrived at the correct numeric value but relied on built‑in elliptic integral functions and omitted the symbolic derivation.

Implications and Outlook

The authors argue that scientific research naturally provides both frontier problems and rich verification mechanisms (proofs, simulations, tool execution). By embedding task generation, distribution, and evaluation into a single optimizable system, BigBang‑V1 demonstrates early recursive self‑improvement at the data layer. While not a full recursive self‑improvement (RSI) system, it showcases a practical step toward autonomous AI‑driven research pipelines.

Compared with prior efforts such as Absolute Zero and DeepMind’s AlphaEvolve, BigBang‑V1 uniquely integrates the task‑generation program, task distribution, and evaluation criteria into a joint optimization target, calibrated by real scientific tasks. The work suggests that future model progress may increasingly depend on improving the efficiency and sophistication of data‑engine pipelines rather than merely scaling parameters.