Machine Heart

Oct 23, 2026 · Artificial Intelligence

TRACE: Rollout-Guided FP4 Quantization Makes MoE RL Training Fast and Stable, 5.4× Speedup

Alibaba Qwen and Ohio State University propose TRACE, a rollout-guided quantization-aware training method that aligns training-side FP4 rounding with actual rollout quantization results, enabling stable FP4 W/A and KV cache rollout for MoE models with up to 5.4× decoding throughput gains and performance matching BF16 baselines.