Skip to content
Axion 1.0: Training Powerful Agents with Only Two-Stage SFT and RL

Axion 1.0: Training Powerful Agents with Only Two-Stage SFT and RL

ByteDance and university researchers introduce Axion 1.0, an agent training method using just two-stage supervised fine-tuning and reinforcement learning, achieving 59% on Humanity's Last Exam and 62.2% on SEAL-0 with fully open-sourced data and code.

Machine Heart
Machine Heart
Machine Heart

Overview

Axion 1.0 is a new agent foundation model developed by researchers from Duke University, ByteDance, University of Maryland, and SUNY Buffalo. It demonstrates strong scientific reasoning and tool-use capabilities using a streamlined training pipeline: two stages of supervised fine-tuning (SFT) followed by reinforcement learning (RL). The method avoids mid-training, direct preference optimization (DPO), and online policy distillation (OPD). The team fully open-sources data construction, generation, filtering, verification, prompts, and training code.

Methodology

Two-Stage Supervised Fine-Tuning

The training tasks are divided into two categories: (1) reasoning and calculation using only information provided in the problem, and (2) tasks requiring external information retrieval via tools. Each SFT stage targets one category.

Stage 1: Difficulty Enhancement for Internal Reasoning

Seed problems from mathematics, physics, chemistry, biology, and computer science are used. To expand learning space beyond what the base model already solves, the team enhances difficulty based on the base model's performance: answers become intermediate quantities in new problems, fixed values are replaced with parameters, ranges are enlarged to break enumeration strategies, multi-step dependencies are added, and case-analysis conditions are introduced. After each rewrite, reference answers are re-verified; problems with insufficient conditions or ambiguous answers are discarded. Correct solution trajectories are collected, yielding ~10.8k Stage-1 SFT examples. Python is allowed but search and web access are disabled, forcing the model to master internal reasoning and calculation.

Stage 2: Tool-Use Tasks Requiring External Evidence

Tasks are constructed from public structured data. A program selects a unique entity, computes the answer, and a language model rewrites the process into a natural-language question. The reference answer is fixed at creation time for later verification. During trajectory collection, the database tables and reference answers are withheld; the model must use search, web access, and Python to gather information. Questions answerable without retrieval are filtered out. Additional checks ensure entity uniqueness, parameter completeness, and no answer leakage in the rewrite. This yields ~7.5k correct tool-interaction trajectories. Both SFT stages retain full interaction context; loss is computed only on assistant-generated reasoning, tool calls, and final answers.

Reinforcement Learning with Staged Rewards

The 35B and 122B main models (based on Qwen3.6-35B-A3B and Qwen3.5-122B-A10B) undergo asynchronous GRPO in environments with search, web access, academic retrieval, and Python. The key design is a staged reward mechanism enabled by the programmatic task construction: the ground-truth entity and intermediate values are known. Full reward is given for a correct final answer; verified intermediate progress (e.g., correct entity identification, then parameter retrieval, then calculation) earns partial rewards, while failure to submit a final answer incurs a penalty. Later steps only count if prior steps pass verification, and entity identification must be supported by tool observations, preventing accidental credit from retrieval snippets.

Experimental Results

Main Benchmarks

Humanity's Last Exam (HLE): Axion 122B scores 59.0%, surpassing GPT-6 Astra (57.2%) and GPT-5.5 (52.2%), approaching Claude Opus 5 (63.6%) and Claude Fable 5 (63.8%).

FrontierScience Research: 122B achieves 66.7%, exceeding GPT-5.6 Sol (63.3%) and GPT-5.5 (26.7%).

FrontierScience Olympiad: 122B and 35B reach 87.0% and 86.0%, both above GPT-5.6 Sol (79.0%) and Gemini 3.1 Pro (85.0%).

SEAL-0 (information seeking): 35B scores 62.2%, beating GPT-5.6 Sol (53.2%), Gemini 3.1 Pro (53.2%), and Agents-A1 (56.4%).

HiPhO: 122B scores 46.2%, higher than GPT-5.6 Sol (42.9%) and GPT-5.5 (43.3%).

Per-Stage Ablation (35B Model)

Stage-1 SFT lifts FrontierScience Olympiad from 71.0% to 83.0%, showing improved scientific reasoning. Stage-2 SFT adds tool use: BioMysteryBench rises from 17.6% to 35.3%, FrontierScience Research from 48.3% to 58.3%. RL further improves SEAL-0 from 53.2% to 62.2%. Checkpoints illustrate the progression from reasoning to information retrieval to multi-step tool interaction.

Different Base Models

Applying the same two-stage SFT to Qwen3.5-35B-A3B and Qwen3.6-35B-A3B: pre-training gap on HLE-130 is 9.2 percentage points; post-training gap shrinks to 0.8 points. Both reach 35.3% on BioMysteryBench, indicating the method helps weaker bases close the gap.

Smaller Models (4B and 9B)

After two-stage SFT only, the 4B model scores 53.1% on HLE-130 and 81.0% on FrontierScience Olympiad; the 9B model scores 63.1% and 83.0% respectively, with 55.0% on FrontierScience Research. The constructed data remains effective at smaller scales.

Open-Source Resources

Paper: https://arxiv.org/abs/2611.03233 Project page: https://axion1-0.github.io Code: https://github.com/bytedance/axion1_0 Training data: https://huggingface.co/datasets/Axion1-0/axion-training-data Model weights (35B):

https://huggingface.co/Axion1-0/Axion-1.0-35B

Conclusion

Axion 1.0 demonstrates that a powerful agent can be trained with a simple two-stage SFT + RL pipeline. The core lies in carefully designed learning tasks: using the model's own solutions to guide difficulty escalation, constructing tool-use questions from public data with verifiable intermediate steps, and optimizing via staged rewards in real tool interactions. The fully open-sourced artifacts provide a concrete starting point for researchers to reproduce and extend agent training.

Original Source

Signed-in readers can open the original source through BestHub's protected redirect.

Sign in to view source
Republication Notice

This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactadmin@besthub.devand we will review it promptly.

open-sourcereinforcement learningtool usesupervised fine-tuningByteDanceagent trainingscientific reasoningAxion 1.0
Reader feedback

How this landed with the community

Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.