ByteDance's Axion 1.0 Surpasses GPT-5.6 Sol on FrontierScience with 66.7 Score

Benchmark Results

Axion 1.0-122B scores 66.7 on the FrontierScience Research benchmark, exceeding the self-reported 63.3 of GPT-5.6 Sol. In the same table, the 35B variant scores 56.7 and Agents-A1 scores 40.0 .

On the HLE benchmark with restricted web access, the 122B model reaches 59.0 and the 35B model 52.2 . Without restrictions, scores improve to 64.8 (122B) and 59.1 (35B).

Training Methodology

The training pipeline consists of only three stages: two supervised fine-tuning (SFT) stages followed by reinforcement learning (RL). The authors explicitly omit intermediate continued pretraining, preference optimization, and online distillation. The 35B model uses Qwen3.6-35B-A3B as its base; the 122B model builds on Qwen3.5-122B-A10B.

Data Generation Strategy

Instead of stacking many training stages, the team invested effort in creating harder training questions. For problems the base model already solves, they increase difficulty by adding steps, converting constants to symbols, expanding ranges, or splitting into cases with different answers. Reference answers are generated by GPT-5.5 solving independently three times; only answers that agree across all three runs are kept.

For retrieval-augmented questions, a program first filters conditions, selects a unique object by maximum or minimum, computes the target quantity, and then hides the database from the model. Questions that can be answered without retrieval are discarded.

Reinforcement Learning Details

RL training starts from the second SFT stage weights. The reward function assigns 1 point for a correct final answer, partial credit for verified intermediate steps, and a small penalty for failing to produce an answer. Training compute for the two SFT stages: 122B model on 32 GPUs for 7 hours 30 minutes ; 35B model for 2 hours 48 minutes (RL compute not included).

Ablation and Progression

Table 2 shows the 122B model's science score progression: base model 40.0 → after first SFT 56.7 → after second SFT 65.0 → after RL 66.7 . The 35B model scores 62.2 on SEAL-0 and 56.4 on Agents-A1.

Open-Source Release

The 35B model weights are released under Apache 2.0 on Hugging Face, accompanied by the data generation and training code. The paper is available on arXiv ( https://arxiv.org/abs/2611.03233) and the code repository on GitHub ( https://github.com/bytedance/axion1_0).

Figures

}