Machine Learning Algorithms & Natural Language Processing
May 6, 2026 · Artificial Intelligence
Why DeepSeek‑V4’s MFU Drops: Parallel Strategies and Compute‑Communication Overlap
The article dissects DeepSeek‑V4’s shift from dense to MoE models, explains why MFU plummets despite sufficient expert dimensions, and details how a carefully designed GPU parallel strategy—combining DP, ZeRO‑1, PP, EP and the new Waved‑EP kernel—overlaps communication and computation to reclaim throughput on 8‑card NVLink nodes linked by InfiniBand.
DeepSeek V4Expert ParallelGPU Distributed Training
0 likes · 19 min read