Tagged articles

Batch Invariance

3 articles · Page 1 of 1

Aug 5, 2026 · Artificial Intelligence

Advanced AI Infra: Making Large Language Models Produce Deterministic Outputs

This article analyzes why LLM inference often yields nondeterministic results, explains how floating‑point addition order, GEMM tiling, Split‑K, RMSNorm, FlashAttention, and NCCL all contribute to batch variance, and details the engineering steps vLLM takes to enforce batch‑invariant execution across GPUs.

Batch InvarianceDeterminismFlashAttention

0 likes · 52 min read

Advanced AI Infra: Making Large Language Models Produce Deterministic Outputs

Old Zhang's AI Learning

May 31, 2026 · Artificial Intelligence

vLLM 0.22 Release: Production-Ready DeepSeek V4 and Extreme KV Cache Compression

The vLLM 0.22 stable release introduces production‑grade DeepSeek V4 support, massive kernel fusions, up to 10‑20× speedups, Batch Invariance with 28.9% latency gain, a Rust front‑end, multi‑level KV cache offload that can double context length, and broad hardware coverage across NVIDIA, AMD, CPU and RISC‑V, making it a pivotal upgrade for inference infrastructure teams.

Batch InvarianceDeepSeek-V4Inference Optimization

0 likes · 13 min read

vLLM 0.22 Release: Production-Ready DeepSeek V4 and Extreme KV Cache Compression

Machine Learning Algorithms & Natural Language Processing

Apr 28, 2026 · Artificial Intelligence

Why DeepSeek V4 Insists on Batch Invariance—and What It Costs

DeepSeek V4 achieves ultra‑long context, complex training pipelines, and custom high‑performance kernels by enforcing batch invariance, a design that guarantees bit‑wise identical outputs across varying batch shapes but incurs lower GPU utilization, reduced small‑batch speed, and added engineering complexity.

Batch InvarianceDeepSeek-V4GPU utilization

0 likes · 8 min read

Why DeepSeek V4 Insists on Batch Invariance—and What It Costs