May 30, 2026 · Artificial Intelligence

Solving AdamW & Muon Instability: Pion Optimizer Updates Large Models on an Iso‑Spectral Manifold

The Pion optimizer leverages iso‑spectral manifold updates to preserve the spectral norm of weight matrices, eliminating additive‑update instability and enabling stable, efficient training of billion‑parameter LLMs across pre‑training, fine‑tuning, and reinforcement‑learning stages, outperforming AdamW and Muon.

AdamWMuonPion optimizer

0 likes · 14 min read

Solving AdamW & Muon Instability: Pion Optimizer Updates Large Models on an Iso‑Spectral Manifold