BestHub
Sign in
Tagged articles

Long Context Retrieval

3 articles · Page 1 of 1
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Oct 9, 2026 · Artificial Intelligence

DeepSeek's Accuracy Varies by Token Position: Phase Sensitivity in KV Cache Compression

ByteDance's Seed team discovered that DeepSeek-V4 models exhibit periodic accuracy fluctuations every 4 tokens in long-context retrieval, caused by chunked KV cache compression creating phase sensitivity where retrieval performance depends on information position relative to compression window boundaries, a phenomenon mitigated but not eliminated by post-training.

ByteDance SeedDeepSeek-V4KV cache compression
0 likes · 12 min read
DeepSeek's Accuracy Varies by Token Position: Phase Sensitivity in KV Cache Compression
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Feb 24, 2026 · Artificial Intelligence

How COMI Achieves 25‑Point Performance Gains at 32× Compression Using Marginal Information Gain (ICLR 2026)

The COMI framework introduces a marginal information gain metric and a coarse‑to‑fine adaptive compression strategy that preserves relevance and diversity, enabling 32× text compression while boosting downstream QA performance by up to 25 points and doubling inference speed.

Long Context RetrievalMarginal Information Gaincontext compression
0 likes · 7 min read
How COMI Achieves 25‑Point Performance Gains at 32× Compression Using Marginal Information Gain (ICLR 2026)
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Feb 23, 2026 · Artificial Intelligence

How COMI Achieves 32× Compression and Boosts Performance by 25 Points

The COMI framework introduces a marginal information gain metric and a coarse‑to‑fine two‑stage compression strategy that preserves relevance and diversity, enabling 32× context reduction while improving Exact Match on NaturalQuestions by nearly 25 points and more than doubling inference speed.

Long Context RetrievalMarginal Information GainNLP
0 likes · 7 min read
How COMI Achieves 32× Compression and Boosts Performance by 25 Points