Skip to content
Tagged articles

prefill-decode separation

3 articles · Page 1 of 1
Alibaba Cloud Infrastructure
Alibaba Cloud Infrastructure
Oct 15, 2026 · Artificial Intelligence

Day-0 Adaptation of Step 5 Preview on Zhenwu M890 Supernode: 3 Key Optimizations

Alibaba Cloud's Zhenwu M890 supernode (64 chips, 800GB/s interconnect, 9TB unified memory) adapted StepFun's Step 5 Preview (600B sparse MoE, 27B active, 1M context, multimodal) with operator fusion, expert parallel optimization, and prefill-decode separation, achieving lower TTFT, higher decode throughput, and stable 1M context support.

Inference OptimizationMoEStep 5 Preview
0 likes · 6 min read
Day-0 Adaptation of Step 5 Preview on Zhenwu M890 Supernode: 3 Key Optimizations
Code Archaeology
Code Archaeology
Jan 19, 2026 · Artificial Intelligence

LLM Inference Optimization: Metrics, Model & Service Techniques

This article systematically covers LLM inference optimization across performance metrics (TTFT, TPOT, throughput, goodput, MFU, MBU), model-level techniques (quantization, distillation, pruning, speculative decoding, attention optimizations), and service-level strategies (continuous batching, prefill/decode separation, prompt caching, parallelism) to improve latency, throughput, and cost-efficiency.

Inference OptimizationKV CacheLLM inference
0 likes · 14 min read
LLM Inference Optimization: Metrics, Model & Service Techniques
Baobao Algorithm Notes
Baobao Algorithm Notes
Jul 11, 2024 · Artificial Intelligence

Why Separate Prefill and Decode? A Deep Dive into DistServe’s Split Inference Architecture

This article explores the two‑stage LLM inference pipeline, introduces TTFT and TPOT metrics, explains the motivation for prefilling‑decoding separation, presents experimental comparisons between split and merged architectures, and details optimization techniques and parallel‑strategy modeling for DistServe.

DistServeGoodputLLM inference
0 likes · 28 min read
Why Separate Prefill and Decode? A Deep Dive into DistServe’s Split Inference Architecture