Day-0 Adaptation of Step 5 Preview on Zhenwu M890 Supernode: 3 Key Optimizations

Model and Hardware Overview

Step 5 Preview is StepFun's flagship agent-oriented model using a sparse Mixture-of-Experts (MoE) architecture with 600B total parameters and 27B activated parameters. It supports a 1M token context window and multimodal input (text, image, video). The model excels at repository-level software engineering, frontend/visualization development, long-horizon agent execution, and financial scenarios requiring quantitative reasoning and large-scale information analysis.

The Lingjun Zhenwu M890 supernode instance is built on T-Head's Zhenwu M890 training-inference integrated AI chips. Each instance interconnects 64 chips via the ICN Switch 1.0 at 800GB/s All-to-All bandwidth with unified memory addressing, providing 9TB of aggregate VRAM. The chips natively support data precisions from FP32 down to FP4.

Three Key Optimization Areas

1. Long-Context and Multimodal Operator Fusion

Fine-grained operators such as RMSNorm, RoPE, and quantization/dequantization were horizontally fused to significantly reduce kernel launch overhead and memory access bottlenecks. For multimodal processing, cross-modal interaction operators between the vision encoder and projection layer were structurally merged, reducing redundant frame-level computation during image-text concatenation.

2. Expert Parallel (EP) Strategy Adjustment and Optimization

Targeting MoE sparse routing characteristics, the team replanned expert distribution across GPUs and optimized communication topology. All-to-All communication scheduling was tuned to reduce expert computation bubbles and cross-GPU synchronization waits. Differentiated parallel strategies were adopted for prefill and decode phases: prefill uses large-batch aggregation to trade communication for compute density, while decode introduces a low-latency All-to-All path to improve parallel resource utilization.

3. Inference Framework Deployment Architecture Optimization

A prefill-decode disaggregated deployment was implemented. KV Cache is transferred across nodes via the high-speed interconnect, allowing independent scaling of prefill and decode throughput. Combined with Multi-Token Prediction (MTP) speculative sampling, single-token generation latency is reduced while maintaining output quality.

Performance Results

After joint optimization, Step 5 Preview runs successfully on the Zhenwu M890 supernode with significant gains: first-token latency (TTFT) is substantially reduced, per-GPU decode throughput (tokens/s) improves by multiple times, and the system stably supports the full 1M context length for long-document understanding, complex reasoning, and long-horizon agent execution.

Commercial Context and Future Direction

This adaptation reflects full-stack collaboration across chip, inference framework, and cloud platform among Alibaba Cloud, T-Head, and StepFun. The Zhenwu M890 supernode, claimed as China's first large-scale commercialized supernode product, offers better cost-performance than flagship GPUs and has served 100+ customers in 40 days across large models, autonomous driving, and embodied intelligence. At the recent Yunqi Conference, Alibaba previewed the next-generation Zhenwu V900 supernode with 3x performance improvement, aiming to provide an open software stack and domestically accessible supernode compute for sparse MoE flagship models like Step 5 Preview. The three parties plan to deepen cooperation on model adaptation, performance tuning, and production deployment.