Machine Heart
May 29, 2026 · Artificial Intelligence
Beyond TurboQuant: Introducing a True 2‑bit KV Quantization for Long‑Context LLM Inference
OSCAR, a new attention‑aware 2‑bit KV cache quantization method, cuts KV memory by up to 8×, delivers up to 3× decode speedup and 7× throughput gain, and matches BF16 accuracy across 4B‑32B models on diverse long‑context reasoning tasks, surpassing TurboQuant.
2-bit compressionKV cacheLLM Quantization
0 likes · 12 min read