DeepSeek‑V4‑Lite‑285B Hits 100% Recall in 256K Token Tests – A Needle‑in‑a‑Haystack Benchmark

Recent community reports indicate that DeepSeek is testing a new model, likely the DeepSeek‑V4‑Lite‑285B, evaluated with the OpenAI MRCR 8‑pin benchmark to assess information retrieval performance in ultra‑long contexts.

In the 128K‑token test, multiple samples achieved outstanding scores:

Index 32: 132,828 tokens, score 1.0000 ✅

Index 45: 135,258 tokens, score 1.0000 ✅

Index 78: 130,660 tokens, score 0.9821 🆗

For the 256K‑token test, a sample also reached perfect performance:

Index 97: 253,819 tokens, score 1.0000 ✅

The model name in the test was a placeholder, likely to avoid premature disclosure of the actual version.

The forum‑shared results indicate that the model attains a 100% recall rate in a native 256K‑token context, but the recall stabilizes around 60% when the context length exceeds this range.

A user warned that the "needle‑in‑a‑haystack" testing method might be gamed by DSA (a possible special mechanism), which can directly recognize specific markers used in the test pattern, suggesting that the impressive scores may not fully reflect real‑world performance.

Although the model has not yet been updated on HuggingFace, the community remains highly interested in DeepSeek's progress on long‑context handling, as achieving a million‑token effective window would represent a significant breakthrough for open‑source models.

The above information is unverified and provided for reference only.