Machine Learning Algorithms & Natural Language Processing

Oct 28, 2026 · Artificial Intelligence

BaGuaLu-2: Runtime-Adaptive Memory Management for MoE Training on 10K+ GPU Clusters

Tsinghua University's BaGuaLu-2 system dynamically selects per-operator recomputation or offloading after MoE gating to keep activation memory within device limits, achieving 1.14× throughput over baselines on 12,288 GPUs with 32.9% average MFU, zero config changes, and open-source code.