DeepSeek V4.1 Flash introduces a new asymmetric Causal Encoder-Decoder architecture with 552B MoE parameters, reducing KV cache HBM demand by 75% and SSD demand by 87.5% while adding native multimodal vision and speculative decoding.
The article reviews DeepSeek‑V4‑Flash‑Vision‑Exp, exposing a misidentification bug, detailing its visual‑primitive approach, impressive spatial‑reasoning benchmarks, and a highly compressed KV‑cache architecture that balances performance with efficiency.
Traditional full‑attention cannot handle million‑token contexts due to exponential compute and memory growth, but DeepSeek V4’s Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA) compress, sparsely index, and precisely compute tokens, cutting KV cache to 10% and FLOPs to 27% while enabling a 1‑M token window on a single GPU.
ClusterAttn tackles the KV‑cache bottleneck of large language models by exploiting the natural clustering of attention scores, achieving up to 92% compression without accuracy loss, boosting throughput 2.6–4.8×, handling 128K‑token sequences on a single GPU, and outperforming existing training‑free compression methods.
