How to Speed Up Claude Code by 20× Without Losing Quality – Cognition’s Fast Context Breakthrough

Problem Statement

Current coding agents like Claude Code spend a large portion of time (about 60%) on context retrieval, leading to total task durations of several minutes, while actual code generation takes only seconds.

Traditional Retrieval Methods

Embedding Search (RAG) : Fast after indexing but often inaccurate for multi‑hop code queries and can return irrelevant embeddings that mislead the agent.

Agentic Search : Uses CLI tools and mimics human behavior; however it requires many sequential tool calls, causing high latency and context pollution.

Cognition’s Fast Context Solution

Cognition introduced two models, SWE‑grep and SWE‑grep‑mini , designed for high‑parallel context retrieval while maintaining state‑of‑the‑art retrieval quality.

(1) Save Lead‑Agent Context Budget

By delegating retrieval to sub‑agents, the main LLM (the lead‑agent) conserves tokens and avoids context pollution, focusing only on relevant information.

(2) Context Retrieval Throughout the Task

Fast Context sub‑agents provide concrete file‑list results with line ranges instead of textual summaries, enabling objective reward signals for reinforcement learning.

(3) Verifiability

Retrieval tasks are verifiable, allowing scalable RL training with clear reward functions.

Key Engineering Techniques

Parallel Tool Calls

Traditional agentic search needs 10‑20 sequential rounds. Cognition’s approach reduces this to four rounds by executing up to eight tool calls in parallel per round, achieving deep‑search in seconds.

Fast Tool Execution

Tool calls are optimized (indexing, multithreading, constrained tool sets) to minimize latency.

Fast Inference

Through collaboration with Cerebras, SWE‑grep reaches >2,800 tokens/s, SWE‑grep‑mini >650 tokens/s, which is 20× and 4.5× faster than Claude Haiku 4.5 (140 tokens/s).

Reinforcement‑Learning Training

Training uses a policy‑gradient objective where the reward R is the weighted F1 score of file‑line retrieval. Importance sampling is applied per‑sequence to obtain unbiased gradient estimates: ∇θ J(θ) = E[∇θ log πθ(a|s) * (R - b(s))] To reduce variance, a leave‑one‑out baseline and a constant scaling factor are incorporated.

Instability Mitigations

Mask loss terms from overly long trajectories.

Mask loss terms from excessively large importance‑sampling ratios.

Remove format‑related reward signals.

Zero‑reward and terminate trajectories with malformed tool calls.

Scale advantage values by average tool‑call count per round.

Data and Evaluation

Cognition built an internal dataset called Cognition CodeSearch Eval containing real‑world repositories, user queries, and manually annotated relevant files and line ranges.

Evaluation metrics:

Weighted F1 (β=0.5) emphasizing precision to avoid context pollution.

End‑to‑end latency.

Models were limited to four rounds with up to eight parallel tool calls per round.

Results

Both SWE‑grep and SWE‑grep‑mini match or exceed state‑of‑the‑art retrieval quality while delivering an order‑of‑magnitude speed improvement.

Downstream Analyses

When integrated as a sub‑agent in larger pipelines (e.g., Windsurf’s Cascade agent), Fast Context reduced total latency while completing the same number of tasks.

In code‑base Q&A benchmarks, SWE‑grep showed lower latency compared to Claude Coder and Cursor CLI.

Concluding Insights

The study highlights the trade‑off between speed and intelligence, demonstrating that highly parallel sub‑agents can break the traditional speed‑intelligence barrier for coding tasks. However, the authors caution that overly fast but “brain‑less” agents are undesirable, and the optimal design lies between lightweight millisecond‑scale agents and fully autonomous, slower agents.