Machine Learning Algorithms & Natural Language Processing

Apr 4, 2026 · Artificial Intelligence

How Gram‑Newton‑Schulz Halves Muon Optimizer’s Compute Cost for Trillion‑Parameter Models

The article explains how the Muon optimizer’s expensive Newton‑Schulz orthogonalization is accelerated by the Gram‑Newton‑Schulz algorithm, which reduces end‑to‑end orthogonalization time by 40‑50%, achieves up to 2× speed‑up in large‑scale LLM training, and resolves numerical stability issues through a restart strategy and custom GPU kernels.