vLLM 0.32.0 Adds System 1 Decision Engine for Millisecond Structured Decisions

System 1 Decision Engine: Fast Structured Decisions

vLLM 0.32.0 adds a new POST /v1/systemone endpoint that implements a "System 1" fast-decision mode inspired by Kahneman's dual-process theory. Unlike traditional autoregressive decoding (System 2), this endpoint performs a single forward pass to return structured decisions with confidence probabilities.

Supported Question Types

choice (multi-class classification): selects from a candidate set and returns probability distribution (e.g., billing 95.5%, shipping 3.2%, security 1.3%).

noul (boolean verdict): returns Yes/No probability directly.

score (ordinal scoring): outputs expected score, level names, and probability distribution across 2–10 tiers.

Integrated Models

The endpoint natively supports mainstream autoregressive models (e.g., Qwen3) via single-label token reading, and integrates specialized decision models: LiquidAI/d1-3B (Liquid AI) clef-omni (Cloudflare)

DiffusionGemma

Benchmark: Snake Game Demo

Official demo runs a real-time Snake game where each move is an HTTP request to /v1/systemone with LiquidAI/d1-3B:

666 decisions in 15 seconds

44 decisions/second average

Median latency: 23 ms

46 apples eaten, 1 death

JevBench results show strong performance on noul and score tasks across models.

Quick Start

pip install vllm
vllm serve LiquidAI/d1-3B
# or
vllm serve Qwen/Qwen3-0.6B

Example request:

curl -s localhost:8000/v1/systemone \
 -H "Content-Type: application/json" \
 -d '{
   "model": "Qwen/Qwen3-0.6B",
   "state": {"team": "billing", "language": "French"},
   "questions": {
     "team": {
       "type": "choice",
       "instructions": "Which team does the message name?",
       "criteria": {"billing": null, "shipping": null, "security": null}
     },
     "lang": {
       "type": "choice",
       "instructions": "Which language does the message name?",
       "criteria": {"English": null, "French": null}
     }
   },
   "chat_template_kwargs": {"enable_thinking": false}
 }'

Response returns clean structured decisions with probabilities:

{
  "id": "decision-80bd0a693f255e1e",
  "object": "structured_decision",
  "model": "Qwen/Qwen3-0.6B",
  "answers": {
    "team": {
      "type": "choice",
      "choice": "billing",
      "probabilities": {"billing": 0.955, "shipping": 0.032, "security": 0.013}
    },
    "lang": {
      "type": "choice",
      "choice": "French",
      "probabilities": {"French": 0.988, "English": 0.012}
    }
  }
}

Low-Precision KV Cache Improvements

NVFP4 KV cache backported to SM8x (A100, RTX 3090/4090) and SM90 (H100), no longer limited to Blackwell SM100.

GLM-5.x defaults to FP8 KV on SM100, halving per-GPU KV memory and boosting throughput 2.3%–5.5%.

GLM-5.3-Flash throughput +31% on SM90 with fp8_ds_mla option.

AMD ROCm 4-bit KV cache via ultraquant_4bit, doubling storage density vs FP8.

Universal HF Tool & Reasoning Parsers

New flags --tool-call-parser hf and --reasoning-parser hf automatically extract response_template from Hugging Face model configs, enabling zero-code tool/reasoning parsing for both Python and Rust frontends.

Cold-Start & Concurrency Optimizations

vllm download-kernels

fetches precompiled FlashInfer kernels, eliminating JIT compile wait at startup. vllm preload --preload-autotune persists autotune cache across restarts.

Shared-Memory Tensor Arena (enabled by default) copies large multimodal tensors once across processes.

With --api-server-count > 1, least-load routing boosts short-text concurrent throughput 43%–85%.

Breaking Changes

Python 3.10 dropped ; minimum is now Python 3.11.

CORS no longer allows all origins by default ; set --allowed-origins explicitly for web clients.

Dependency upgrades : transformers>=5.16.1, tokenizers>=0.23.2.

max_tokens semantics changed : now treated as output ceiling, not pre-allocated memory quota; long prompts no longer rejected early.

Cascade attention removed ; --disable-cascade-attn deprecated and will be removed next release.

Conclusion

vLLM 0.32.0 expands the engine from pure autoregressive generation to low-latency high-frequency structured decision-making. The /v1/systemone endpoint paired with lightweight decision models addresses Agent routing latency and memory pressure. Precompiled wheels and Docker images are available; upgrade recommended.