Skip to content
Tagged articles

expert parallel

2 articles · Page 1 of 1
Alibaba Cloud Infrastructure
Alibaba Cloud Infrastructure
Oct 15, 2026 · Artificial Intelligence

Day-0 Adaptation of Step 5 Preview on Zhenwu M890 Supernode: 3 Key Optimizations

Alibaba Cloud's Zhenwu M890 supernode (64 chips, 800GB/s interconnect, 9TB unified memory) adapted StepFun's Step 5 Preview (600B sparse MoE, 27B active, 1M context, multimodal) with operator fusion, expert parallel optimization, and prefill-decode separation, achieving lower TTFT, higher decode throughput, and stable 1M context support.

Inference OptimizationLong ContextMoE
0 likes · 6 min read
Day-0 Adaptation of Step 5 Preview on Zhenwu M890 Supernode: 3 Key Optimizations
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
May 6, 2026 · Artificial Intelligence

Why DeepSeek‑V4’s MFU Drops: Parallel Strategies and Compute‑Communication Overlap

The article dissects DeepSeek‑V4’s shift from dense to MoE models, explains why MFU plummets despite sufficient expert dimensions, and details how a carefully designed GPU parallel strategy—combining DP, ZeRO‑1, PP, EP and the new Waved‑EP kernel—overlaps communication and computation to reclaim throughput on 8‑card NVLink nodes linked by InfiniBand.

GPU Distributed TrainingMoEPipeline Parallel
0 likes · 19 min read
Why DeepSeek‑V4’s MFU Drops: Parallel Strategies and Compute‑Communication Overlap