How Huawei Cloud Optimizes the New Xiaohongshu Open‑Source Model dots3‑note Preview for Multimodal Inference

Model Architecture

dots3-note preview is the first model in the dots3 series, built with 280 B total parameters and 16 B activation parameters. It supports text, vision, and audio modalities. The model uses a mixed sparse attention mechanism combining SWA (Sliding Window Attention) and DSA (Dynamic Sparse Attention) on the MLA base, extending KV‑Cache length to 512 k with minimal overhead. A more sparse MoE architecture is employed, adding expert‑level and sequence‑level balance loss terms to mitigate load imbalance.

Training Innovations

Text pre‑training : Gated attention stabilizes training. UltraEP, a soft‑hard co‑balancing system for fine‑tuned MoE, performs dynamic load balancing based on runtime routing, improving training efficiency.

Multimodal pre‑training : The BigMac pipeline scheduling system coordinates heterogeneous vision‑language workloads via dependency‑safe nested pipelines, reducing pipeline bubbles and activation memory.

Post‑training : A unified reward mechanism aligns supervised‑fine‑tuning (SFT) and reinforcement‑learning (RL) objectives; a group‑wise length‑ranking reward eliminates length bias, increasing generation speed while preserving alignment.

Long‑range agent capability : Reinforcement‑learning methods target ultra‑long tasks, leveraging strong multimodal visual understanding for tasks such as invoice image organization, aesthetic design alignment, and precise visual element localization.

Inference Optimizations

Speculative decoding : Native support for MTP speculative decoding in vLLM Ascend generates multiple token candidates per forward pass, reducing token‑per‑output time (TPOT) and improving throughput.

FlashComm optimization : Redundant AllReduce operations for RMSNorm, Dynamic Quant, and MLA QKV reduction are transformed into ReduceScatter + AllGather, moving column‑independent operators to a two‑stage communication phase, eliminating duplicate calculations and lowering latency.

FUSED_MC2 fusion : The dispatch_ffn_combine fusion operator merges dispatch communication, GMM, SwiGLU, and combine steps into a single kernel, cutting kernel launches and deepening compute‑communication pipelines, significantly raising MoE inference throughput.

MLA path upgrade : Q/K/V head dimension is expanded from 512 to 1024, supporting kv_lora_rand = 1024 in SWA layers; SWA window specifications are broadened to cover longer sequences, enhancing decode throughput and long‑sequence generation.

Deployment on Huawei Cloud

CloudMatrix AI service adapts dots3-note preview using the vLLM Ascend inference engine, preserving end‑to‑end multimodal capabilities (text, image, audio) and delivering stable inference for diverse input scenarios. Optimizations include:

Full‑modal end‑to‑end integration via vLLM Ascend, linking visual encoders, audio feature extractors, and the LLM backbone.

FlashComm communication restructuring to eliminate redundant cross‑GPU calculations.

FUSED_MC2 kernel fusion for MoE layers.

MLA Q/K/V dimension upgrade and expanded SWA window support.

Deployment instructions are available at:

https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/Dots3-Note.html