Skip to content
Tagged articles

Prefill Decode

4 articles · Page 1 of 1
Alibaba Cloud Native
Alibaba Cloud Native
Oct 20, 2026 · Artificial Intelligence

Inference Frameworks Demystified: Core Concepts, Frameworks & Cluster Deployment for LLMs

This comprehensive guide explains inference frameworks for large language models, covering core concepts like Prefill/Decode and KV Cache, comparing major open-source frameworks (vLLM, SGLang, TensorRT-LLM), detailing request processing flow from scheduling to sampling, and identifying who benefits from understanding inference infrastructure.

KV CacheLLM servingMoE
0 likes · 46 min read
Inference Frameworks Demystified: Core Concepts, Frameworks & Cluster Deployment for LLMs
Machine Heart
Machine Heart
Jul 3, 2026 · Artificial Intelligence

Avoiding Pitfalls in Heterogeneous Token Factories: Industry‑Level Design Practices for Cross‑Hardware LLM Inference

The article analyzes a recent multi‑institution paper that maps the design space of heterogeneous Prefill‑Decode LLM inference, identifies three core boundary decisions, presents nine deployment best practices, and validates them with a production token‑factory case on MuXi C600 and NVIDIA Hopper GPUs.

KV CacheLLMPrefill Decode
0 likes · 11 min read
Avoiding Pitfalls in Heterogeneous Token Factories: Industry‑Level Design Practices for Cross‑Hardware LLM Inference
Huawei Cloud Developer Alliance
Huawei Cloud Developer Alliance
Apr 29, 2026 · Artificial Intelligence

Deploy DeepSeek‑V4 on Ascend NPU with Kthena in 3 Minutes (Prefill‑Decode Separation)

This guide walks through deploying the DeepSeek‑V4‑Flash model on Ascend NPU using Kthena’s ModelRoute, detailing the Prefill‑Decode (P/D) separation architecture, KV cache transfer via Mooncake, configuration of ModelServing and ModelRoute resources, and flexible scaling of Prefill and Decode replicas for optimal performance.

Ascend NPUKV CacheKthena
0 likes · 22 min read
Deploy DeepSeek‑V4 on Ascend NPU with Kthena in 3 Minutes (Prefill‑Decode Separation)