Skip to content
Tagged articles

Random Attention

2 articles · Page 1 of 1
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Sep 21, 2026 · Artificial Intelligence

Random KV Cache Eviction Rivals Top Baselines, Boosts Throughput 43%

Salesforce AI Research and UIUC propose Random Attention, a simple KV cache eviction method that randomly retains reasoning tokens while protecting the prompt, matching state-of-the-art baselines across math, science, and code reasoning tasks and improving inference throughput by 32–43% in vLLM serving.

Cache EvictionKV CacheLLM inference
0 likes · 20 min read
Random KV Cache Eviction Rivals Top Baselines, Boosts Throughput 43%
Machine Heart
Machine Heart
Sep 19, 2026 · Artificial Intelligence

Random Attention: Random KV Cache Eviction Rivals Best Baselines, Boosts Throughput 43%

Salesforce and UIUC researchers propose Random Attention, a simple KV cache eviction method that randomly retains reasoning tokens while fully protecting the prompt, matching the accuracy of complex importance-based methods across multiple models and tasks while increasing inference throughput by 32-43% in vLLM serving.

Attention MechanismCache CompressionEfficient AI
0 likes · 18 min read
Random Attention: Random KV Cache Eviction Rivals Best Baselines, Boosts Throughput 43%