FireFlash: Open Recipe for Long-Horizon Agent RL Matches Frontier at 30x Lower Cost

Overview

FireFlash is a 300B-parameter long-horizon agent for terminal and programming tasks, obtained by full-parameter reinforcement learning (RL) starting from GLM-5.3-Flash (321B total parameters, 17.4B active per token). Training used 64 NVIDIA B300 GPUs and completed in under five days. The model reaches 46.2% Pass@1 on Terminal-Bench 4.0, on par with Fable 5 and GPT-6 Sol, while its per-task inference cost is over 30× lower than Fable 5 at maximum inference intensity. Gains transfer to DeepSWE v1.1 (+5.4 points) and SWE-Bench Pro Verified (+5.1 points). The team publishes the complete training recipe on the Agent Economics blog (https://agenteconomics.org/posts/agentic-rl-recipe).

Infrastructure: Fully Asynchronous RL

Core insight: idle GPUs cost more than stale rollouts—let sampling and training run at full speed independently.

Separate GPUs for training and rollout. Half the GPUs train with the slime framework in BF16; the other half run the SGLang inference engine with weights and KV cache in FP8. After each training step, new weights are synced via NCCL to the rollout engine; training and sampling never block each other.

Training on stale rollouts. Because sampling runs continuously, a turn may be generated by weights several steps behind the trainer. Rollouts from weights more than six versions old are discarded. Policy staleness introduces minimal shift: log-probability gap between trainer and rollout is 0.065 at one version stale and only 0.08 at four to six versions stale.

Fast, isolated rollouts. The model's native multi-token prediction (MTP) head enables speculative decoding, speeding up decoding by 1.38×. Each rollout and scoring run inside an independent Firecracker microVM forked from a parent VM in under 100 ms. Infrastructure-related rollout-group loss dropped from 12% to 0.5%.

Task Selection: Train Only Where the Model Is Uncertain

Core insight: spend rollouts only on tasks the model is unsure about, and measure that uncertainty under exactly the same conditions as training.

GRPO learns from relative differences among multiple attempts on the same task. Tasks that are always solved or never solved consume a full rollout group but provide zero learning signal. The team probes each candidate task with at least four rollouts using the base model, keeping only those with initial pass rates between 15% and 85%. Probing strictly uses the training configuration—same harness, inference intensity, context window, turn budget, and sandbox—because pass rates depend on every setting. This filtering reduced zero-signal rollout groups from 48% to 10.2%. Final training data comprises ~52% long-horizon agent tasks (custom long-horizon tasks 30%, hard variants 11%, long-horizon coding skills 9%, public MiMo-TB and Tasksmith 1% each) and ~48% software-engineering tasks (public MiMo-code 18%, custom SWE tasks: Python-heavy 18%, multi-language 12%).

Reward Design: Verify Scorers Before Preventing Reward Hacking

Core insight: tasks the model never solves are often broken tasks—verify every scorer before rewards reach the model.

Reviewing never-solved tasks revealed nearly half were flawed: defective problem statements, brittle tests that misjudge correct solutions, or "protected file" traps. Some scorers awarded partial points merely for matching output format or crashed on real agent outputs. To eliminate these, every new task must pass nine validation gates before entering the training pool:

Task (2 gates): reference solution passes in the training sandbox under the exact training pipeline; every scorer requirement is stated in the problem description.

Sandbox (3 gates): runs offline; reliably starts every time without hanging; no answer leakage.

Scorer (4 gates): do-nothing or boilerplate attempts score 0; tests are not brittle—any correct solution passes; even malformed real outputs are scored without crashing; identical outputs always receive identical scores.

Reward shaping. Each attempt receives a terminal reward in [0,1]: 1.0 for full solve; partial credit for passed verification checks (SWE tasks scored by fixed failed tests). Solved attempts earn an additional length reward up to 0.2, proportional to tokens saved relative to the group's longest solution. This shaping reduced tokens per solve to 0.84× the original count without lowering evaluation accuracy.

Harness: Built for Long-Horizon Attempts, Training/Eval Identical

Core insight: the harness is part of the model—whatever harness evaluates, the same harness must train.

The harness follows Claude Code conventions: four tools (Bash, Read, Write, Edit), native tool calling, retained reasoning history, and context compression. Crucially, training rollouts and evaluation trials are driven by the exact same harness, with automated unit tests guaranteeing code and prompt parity. Before any RL, this harness alone lifted GLM-5.3-Flash 4.4 points above its official Claude Code score on Terminal-Bench 4.0.

Long-horizon attempts must stay affordable. Stateful context compression triggers at 85% context-window usage, preserving prefix cache by appending sectional summaries. Commands write stdout to files, returning only truncated previews with filenames. Long-running commands execute asynchronously in the background instead of timing out. These features together cut per-trial cost by ~33%. An additional prompt specification for efficient workflows (in-place edits, single writes, no sleep polling) saves another 16%.

RL Objective: Learn from Stale, Low-Precision Samples Without Drift

Core insight: correct stale, low-precision samples instead of discarding them, and treat entropy rise as an early warning.

Rollouts are sampled in FP8 on weights up to six versions behind the BF16 training policy. Rather than discarding these stale trajectories, the team corrects them directly in the training objective. The core algorithm is GRPO with 12 attempts per task and one optimizer step per batch. Each attempt's advantage combines a reward term and a length term:

A = (R′ − mean R′) / std R′ + (b − mean b) / 0.5

(mean and std computed over the 12 attempts for the same task) R′ = 1.5 if the attempt solves the task; otherwise R′ = 0.5 × R (where R is the terminal reward in [0,1]).

b = 0.2 × (1 − tokens / max tokens among solved attempts in group)

, computed only when at least two attempts solve; otherwise 0.

Truncated Importance Sampling (TIS). Each token is weighted by the ratio of BF16 training-policy probability to FP8 rollout probability, clipped to [0.5, 2.0] to bound single-token influence. Only 1%–3% of tokens hit the clipping boundaries.

Score Centralization (SC). The rollout engine returns top-32 log-probabilities per token. The trainer subtracts the sampler's expected score over these candidates from each token's policy-gradient score. This eliminates residual drift toward the stale FP8 sampler that clipping alone cannot remove, at a 7.6% per-step time overhead.

Success Offset. Full solves (R=1.0) enter the advantage as 1.5; unsolved attempts contribute half their partial credit: the best "near miss" (R=0.5) counts only 0.25, while a solve counts 1.5. Partial credit shapes learning but never equals a true solve.

Loss Normalization Floor. Each attempt's loss is normalized by its token count, with a minimum denominator of 2,048 tokens, preventing degenerate short rollouts from dominating the gradient.

With these corrections, training uses neither KL penalty nor entropy bonus . In healthy runs, policy entropy slowly declines from ~0.46 to ~0.33; an entropy increase serves as an early instability warning.

Experiments: Across-the-Board Gains on the Cost–Performance Frontier

FireFlash is evaluated in the same harness used for training, compared against GLM-5.3-Flash official scores and other models' public results.

FireFlash improves over its base on all three benchmarks: Terminal-Bench 4.0 +10.4 points, surpassing Fable 5 and the full-size sibling GLM-5.3; DeepSWE v1.1 +5.4 points, surpassing Fable 5.1; SWE-Bench Pro Verified +5.1 points, surpassing Kimi K3 and GPT-5.6 Sol.

Cost vs. Performance

On Terminal-Bench 4.0, FireFlash sits on the cost–performance Pareto frontier. Among 40+ model/inference-intensity settings, only four have lower per-trial cost, and none exceed 35.8% Pass@1. At GLM-5.3-Flash public pricing, FireFlash costs $0.72 per trial versus $22.02 for Fable 5 at maximum inference intensity. The next frontier point, GPT-6.1 Terra, scores 55.6% at $1.49 per trial. Every model scoring higher than FireFlash costs at least twice as much per trial; the full-size GLM-5.3 at max intensity costs $8.27 per trial for only 41.8%.

Conclusion

FireFlash demonstrates that targeted post-training can turn an open-weight model into a strong domain-specific agent, achieving frontier performance at a fraction of inference cost. The authors argue that jointly optimizing model, training environment, harness, and inference system is a viable path to powerful, efficient agents beyond mere scaling. FireFlash is a first step; the team plans to extend this methodology to more complex, longer-horizon, repeatedly executed work beyond coding and terminal tasks.