Machine Learning Algorithms & Natural Language Processing

Oct 9, 2026 · Artificial Intelligence

DeepSeek's Accuracy Varies by Token Position: Phase Sensitivity in KV Cache Compression

ByteDance's Seed team discovered that DeepSeek-V4 models exhibit periodic accuracy fluctuations every 4 tokens in long-context retrieval, caused by chunked KV cache compression creating phase sensitivity where retrieval performance depends on information position relative to compression window boundaries, a phenomenon mitigated but not eliminated by post-training.