OSCAR Beats TurboQuant: 2‑Bit KV‑Cache for Fast, Stable Long‑Context Inference

OSCAR presents an attention‑aware rotation scheme that compresses KV caches to true 2‑bit, cutting memory usage by up to 8× and boosting decode throughput by up to 7×, while preserving inference quality within a few points of BF16 across multiple models and long‑context benchmarks, outperforming TurboQuant.

2-bit quantizationKV cacheOSCAR

0 likes · 13 min read

OSCAR Beats TurboQuant: 2‑Bit KV‑Cache for Fast, Stable Long‑Context Inference

Machine Heart

May 29, 2026 · Artificial Intelligence

Beyond TurboQuant: Introducing a True 2‑bit KV Quantization for Long‑Context LLM Inference

OSCAR, a new attention‑aware 2‑bit KV cache quantization method, cuts KV memory by up to 8×, delivers up to 3× decode speedup and 7× throughput gain, and matches BF16 accuracy across 4B‑32B models on diverse long‑context reasoning tasks, surpassing TurboQuant.

2-bit compressionKV cacheLLM Quantization

0 likes · 12 min read

Beyond TurboQuant: Introducing a True 2‑bit KV Quantization for Long‑Context LLM Inference

attention-aware rotation

OSCAR Beats TurboQuant: 2‑Bit KV‑Cache for Fast, Stable Long‑Context Inference

Beyond TurboQuant: Introducing a True 2‑bit KV Quantization for Long‑Context LLM Inference