vLLM 0.25 Release: New Model Support, Multi‑Hardware, and Major Performance Boosts

vLLM 0.25 has been officially released, bringing a comprehensive set of enhancements that make it the most capable engine for locally deploying large language models.

New model support

MiMo‑V2.5 – Xiaomi's latest vision‑language model

Laguna XS.2 – new architecture with DFlash support

Moondream3 – lightweight multimodal model supporting query and caption

Qianfan‑OCR – Baidu's Qianfan OCR model

Cohere MoE – Cohere's mixture‑of‑experts architecture

DeepSeek V4 – adds AMD/ROCm support and pipeline parallelism

Qwen3.5 – Mamba hybrid architecture, Model Runner V2 support

Notably, DeepSeek V4 can now run on AMD GPUs, which is a welcome option for users who prefer to avoid NVIDIA hardware.

Multi‑hardware platform expansion

AMD ROCm 7.2.2 – DBO dynamic batch optimization, fused Allreduce+RMSNorm, fused shared expert

CPU – FP8 attention for AMX/AVX‑512, FP8 W8A16 linear/MoE, RISC‑V support

Intel XPU – top‑k/top‑p sampling, out‑of‑place all‑reduce, LoRA

IBM Power – VSX attention backend

From NVIDIA to AMD, Intel, IBM and even RISC‑V, vLLM aims to be a "full‑platform inference champion".

Thinking‑budget aware speculative decoding

Models such as DeepSeek‑R1 and Qwen3 use a "thinking budget" during generation; earlier speculative decoding conflicted with this mechanism, producing incorrect outputs. v0.25 fixes the issue by correctly detecting reasoning‑token boundaries, allowing reasoning models to benefit from speculative‑decoding speedups.

Additional speculative‑decoding support includes:

Gemma4 MTP (Multi‑Token Prediction)

MiMo‑V2.5 MTP

Mistral EAGLE

Cohere Eagle

Performance optimizations: concrete gains

FlashInfer top‑k/top‑p sampler enabled by default – faster sampling

AllPool.forward 51% faster – direct benefit to embedding models

GPU↔CPU synchronization removed – reduces unnecessary waiting

NumPy zero‑copy embedding serialization – faster Embedding API responses

FlashInfer FP8 asynchronous tensor‑parallel fusion – accelerates tensor‑parallel scenarios

Allreduce + RMS fusion re‑enabled – benefits DP/PP workloads

Docker image reduced by 2.5 GB – achieved by lazy downloading of FlashInfer cubin files

KV Cache unload + Hybrid Memory Allocator (HMA)

The existing KV‑cache unload (moving inactive KV entries from GPU to CPU) is now fully integrated with HMA. The integration adds:

Scheduler sliding‑window grouping support

MooncakeStoreConnector for distributed KV unload

DCP/PCP protocol support

This allows serving more concurrent requests with less GPU memory while maintaining high throughput, especially valuable for very long context windows (e.g., 128 K tokens).

Quantization support updates

NVFP4 – KV‑cache support, ModelOpt NVFP4 W4A16, all‑gather GEMM fusion

MXFP4 – Humming MXFP4 MoE backend

TurboQuant – mixed‑model and unified quantization support

NVFP4 receives extensive support; on Hopper or Blackwell GPUs, 4‑bit quantization offers a better performance‑accuracy trade‑off.

API improvements

Responses API – now supports streaming tool calling with required and named tool selection

OpenAI compatibility enhancements: system_fingerprint, prompt_embeds, and rendered prompt text fields

XGrammar 0.2.0 – structured tags for strict tool calling and reasoning

Fastokens tokenizer added

RLHF explicit weight‑update API added

Installation

The current Quickstart assumes Linux and Python 3.10‑3.13. NVIDIA CUDA users are advised to manage the environment with uv:

uv venv --python 3.12 --seed
source .venv/bin/activate
uv pip install vllm --torch-backend=auto

Docker users benefit from the 2.5 GB smaller image, resulting in noticeably faster pulls.

Conclusion

For production‑grade LLM inference services, the author still recommends vLLM as the top choice; its rapid iteration cadence and vibrant community consistently deliver substantive improvements each release.