BestHub
Sign in
Tagged articles

KV Cache Compression

8 articles · Page 1 of 1
Old Zhang's AI Learning
Old Zhang's AI Learning
Oct 17, 2026 · Artificial Intelligence

DeepSeek-V4.1-Flash-1017: Flash Model Matches Flagships on Agent Benchmarks, Cuts KV Cache 4x

DeepSeek releases V4.1-Flash-1017, a flash-tier model that outperforms flagship competitors on key agent benchmarks like DeepSWE and Automation-Bench while reducing KV cache to 1/4 of its predecessor via CSA2 sparse attention and FP4 quantization, and adds native multimodal support, adjustable reasoning intensity, and a new Search & Fetch API for real-time web access.

Agent benchmarksCSA2 sparse attentionDeepSeek
0 likes · 6 min read
DeepSeek-V4.1-Flash-1017: Flash Model Matches Flagships on Agent Benchmarks, Cuts KV Cache 4x
Baidu Intelligent Cloud Tech Hub
Baidu Intelligent Cloud Tech Hub
Oct 15, 2026 · Artificial Intelligence

TB-Scale LLM Inference: Zero-Redundancy Weight Deployment & 80% KV Cache Compression

Baidu Baige presents extreme VRAM optimization for TB-scale model inference: zero-redundancy weight deployment via DP Attn-sharding (releasing 16–30 GiB/card) and full-route CPP (50% throughput gain), plus KV Cache compression via LU-KV (80% reduction, 0.52% accuracy loss), Draft SWA, and PD handshake optimization (20% VRAM release).

KV Cache CompressionLLM inferenceLU-KV
0 likes · 27 min read
TB-Scale LLM Inference: Zero-Redundancy Weight Deployment & 80% KV Cache Compression
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Oct 9, 2026 · Artificial Intelligence

DeepSeek's Accuracy Varies by Token Position: Phase Sensitivity in KV Cache Compression

ByteDance's Seed team discovered that DeepSeek-V4 models exhibit periodic accuracy fluctuations every 4 tokens in long-context retrieval, caused by chunked KV cache compression creating phase sensitivity where retrieval performance depends on information position relative to compression window boundaries, a phenomenon mitigated but not eliminated by post-training.

ByteDance SeedDeepSeek-V4KV Cache Compression
0 likes · 12 min read
DeepSeek's Accuracy Varies by Token Position: Phase Sensitivity in KV Cache Compression
Data STUDIO
Data STUDIO
Sep 11, 2026 · Artificial Intelligence

Running 744B MoE on 25GB RAM: Int4, Disk Streaming & MLA Compression

This article details how a custom C inference engine runs the 744B-parameter GLM 5.2 MoE model on 25GB RAM by quantizing routed experts to int4, streaming them from disk via pread, compressing KV cache 57x with Multi-head Latent Attention, and employing a five-tier memory hierarchy with speculative decoding.

AVX2 kernelsC inference engineGLM-5.2
0 likes · 48 min read
Running 744B MoE on 25GB RAM: Int4, Disk Streaming & MLA Compression
PaperAgent
PaperAgent
Aug 22, 2026 · Artificial Intelligence

DeepSeek’s Hidden Multimodal Model: Technical Deep‑Dive and Unexpected Bugs

The article reviews DeepSeek‑V4‑Flash‑Vision‑Exp, exposing a misidentification bug, detailing its visual‑primitive approach, impressive spatial‑reasoning benchmarks, and a highly compressed KV‑cache architecture that balances performance with efficiency.

DeepSeekKV Cache CompressionMoE
0 likes · 4 min read
DeepSeek’s Hidden Multimodal Model: Technical Deep‑Dive and Unexpected Bugs
Lao Guo's Learning Space
Lao Guo's Learning Space
Apr 30, 2026 · Artificial Intelligence

How DeepSeek V4’s CSA + HCA Break the Million‑Token Barrier

Traditional full‑attention cannot handle million‑token contexts due to exponential compute and memory growth, but DeepSeek V4’s Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA) compress, sparsely index, and precisely compute tokens, cutting KV cache to 10% and FLOPs to 27% while enabling a 1‑M token window on a single GPU.

Attention MechanismCSAHCA
0 likes · 12 min read
How DeepSeek V4’s CSA + HCA Break the Million‑Token Barrier
Network Intelligence Research Center (NIRC)
Network Intelligence Research Center (NIRC)
Dec 23, 2025 · Artificial Intelligence

ClusterAttn: Compressing KV Cache with Intrinsic Attention Clustering

ClusterAttn tackles the KV‑cache bottleneck of large language models by exploiting the natural clustering of attention scores, achieving up to 92% compression without accuracy loss, boosting throughput 2.6–4.8×, handling 128K‑token sequences on a single GPU, and outperforming existing training‑free compression methods.

KV Cache CompressionLarge Language Modelsattention clustering
0 likes · 8 min read
ClusterAttn: Compressing KV Cache with Intrinsic Attention Clustering