Why Claude Code and Cursor Abandoned Vector Databases for Agentic Retrieval

In May 2025, Anthropic quietly removed the entire vector search pipeline — embedding pipeline, local vector database, chunking heuristics — from Claude Code and replaced it with a single grep command. According to Claude Code author Boris Cherny on the Latent Space podcast, the result outperformed everything. By a lot, and this was surprising . A year later, the industry has converged: Cursor hired the engineer behind the decision; Windsurf, Cline, Devin, and Sourcegraph Amp all dropped vector indexes in favor of tool-driven retrieval; Anthropic's own multi-agent research system scored 90.2% higher than a single Claude Opus 4 on internal evaluations; and an Amazon AAAI 2026 paper demonstrated that a keyword-only agent reached 94.5% of RAG's faithfulness with zero vector infrastructure.

01 Why Vector Retrieval Fails on Code

The article first establishes how poorly vector retrieval performs on code tasks. The original SWE-bench baseline (October 2023) used a simple RAG pipeline — chunk codebase, embed, take top‑k, generate patch — and achieved only 1.96% resolution. The first agent system, SWE‑agent, swapped retrieval for tools ( open_file, scroll_down, edit_lines) and jumped to 12.47% . By 2026, the SWE‑bench Verified leaderboard is dominated by agentic systems, none of which rely on vector retrieval.

Five root causes are identified:

Semantic similarity ≠ relevance. The most similar embedding is a poor proxy for "which function breaks if I change this line?" Code has explicit structural relationships (imports, type definitions, call graphs) that flat embeddings flatten.

Identifiers are exact search. When you ask where processPayment is defined, you need precise matching. Vector search yields false positives ( handlePayment) and false negatives (the real definition pushed down by a look‑alike comment).

The index is always stale. Code drifts on every commit; continuous re‑indexing is expensive and never catches up — the same problem GraphRAG faced before LazyGraphRAG.

The index is a liability. A private codebase's vector index is a full copy of that code sitting on separate infrastructure, often with weaker access controls than the source repository.

Single‑shot retrieval is brittle. Top‑k gets one chance; if the first pass misses the right file, the model confidently generates wrong code.

The Amazon paper ( Keyword Search Is All You Need , AAAI 2026) generalizes this failure beyond code across six datasets (FinanceBench, BlockchainSolana, Llama2Paper, HistoryOfAlexnet, etc.). On FinanceBench, agentic keyword retrieval actually beat traditional RAG by 6 percentage points (30.40% vs 24.24%). The chunk‑and‑embed failure mode is universal, not code‑specific.

02 Anthropic's Four Reasons for Dropping Vector Search

Cherny and Cat Wu articulate four reasons for removing vector search from Claude Code:

Accuracy — unexpectedly better. The team expected agentic retrieval to be worse and accepted a quality trade‑off for operational simplicity. Instead, an LLM driving iterative grep can refine queries, read neighboring files, follow imports, and self‑correct — capabilities a single embedding lookup lacks.

Freshness. The agent reads the filesystem directly, reflecting the repository's current state with no index lag. A file edit is visible to Claude Code within ~100 ms; a vector index must wait for the next re‑embedding cycle.

Security & privacy. Cherny: "RAG has a whole indexing step… the index has to live somewhere… that's a huge liability for companies." Enterprise customers especially reject a separate embedded copy of proprietary code on external infrastructure.

Reliability. Fewer components mean fewer failures. grep -based retrieval has no drifting embedding model, no vector database to go down, no re‑indexing pipeline to lag, no chunking strategy to tune. ripgrep, find, cat just work.

Cat Wu summarizes: "Claude is very good at agentic retrieval; you can reach the same accuracy with a much cleaner deployment story."

03 Just‑In‑Time Context Loading: From Pre‑computed Embeddings to On‑Demand Fetch

Anthropic's September 2025 engineering blog Effective context engineering for AI agents names this pattern just‑in‑time (JIT) context loading . It draws a sharp line against traditional RAG:

Traditional RAG = pre‑inference retrieval: everything is pre‑embedded, stored in a vector DB, queried at inference, and top‑k chunks are stuffed into the prompt. The system must predict and index everything the model might need.

JIT loading: the agent maintains lightweight identifiers (file paths, stored queries, web links) and uses tools to dynamically load those references into context at runtime. Nothing is pre‑loaded; the agent fetches exactly what it needs, when it needs it.

This changes the shape of the context window . In pre‑inference RAG, tokens are spent on chunks you guess are relevant ; in JIT loading, tokens are spent only on chunks the agent judges relevant , skipping the rest. Anthropic frames it as "finding the minimal, high‑signal token set that maximizes expected outcome." This also answers the token‑cost critique: naive agent loops do use more tokens than vector RAG, but JIT loading combined with sub‑agent context isolation can use fewer tokens by completely avoiding low‑signal chunks. An internal figure: Claude Code's tool lazy‑loading (loading tool definitions only when needed) reduced context consumption by ~95% .

04 Agent as Retriever: A Deliberately Boring Toolset

In Claude Code, retrieval is exposed as a small set of deliberately boring tools:

Glob — file path pattern matching, near‑zero token cost.

Grep — regex content search via ripgrep (macOS/Linux native builds switched to built‑in ugrep + bfs from April 2026).

Read — load full or partial file content into context.

Bash — fallback shell for long‑tail operations ( tail, head, jq, git log, find with predicates).

Explore subagent — a read‑only independent agent (default Haiku 4.5) with its own context window for parallel codebase exploration.

A reverse‑engineering study of the Claude Code TypeScript source reveals a larger system: 54 built‑in tools (19 unconditional, 35 feature‑gated), yet only 1.6% is AI decision logic ; the remaining 98.4% is operations infrastructure, context management, permissions, tool dispatch, and compression. The decision layer is tiny; the retrieval and context management layer is huge.

The control loop follows:

plan → glob/grep → read candidates → refine query
     → repeat (or spawn subagent) → compact → answer

This mirrors agentic RAG (Self‑RAG, CRAG, A‑RAG) with one critical difference: there is no pre‑built index between the agent and the raw bytes . The retriever is the shell tool the agent chooses to call.

Because JIT loading eventually fills the context window, Claude Code runs a five‑tier compression pipeline when approaching the 200k token limit:

Budget reduction — drop least relevant content first.

Snip — remove redundant tool call outputs.

Microcompact — summarize individual over‑long messages.

Context collapse — fold earlier turns into shorter retrospectives.

Auto‑compact — final summarization when nothing else fits.

This pipeline is the necessary counterpart to JIT loading: you must both load only what you need and gracefully forget what you no longer need.

05 Three Benchmarks That Prove the Point

Amazon's "Keyword Search Is All You Need" (AAAI 2026)

The most rigorous public comparison comes from Subramanian et al. Same LLM (Claude 3 Sonnet, 200k context, temperature 0.001), same six datasets, same evaluation framework; only the retriever differs: one side uses Amazon Bedrock Knowledge Base with Titan Text Embeddings V2, the other a ReAct agent calling pdfmetadata, rga, pdfgrep.

Aggregate results across all datasets:

Faithfulness: agent 0.81 vs RAG 0.86 → 94.5% of RAG.

Context Recall: agent 0.68 vs RAG 0.77 → 88.0% .

Answer Correctness: agent 0.59 vs RAG 0.65 → 91.5% .

The paper concludes: "Vector databases are not necessary for high‑quality retrieval performance. An agentic approach with simple keyword search tools is a viable alternative for many applications."

Search‑R1: Reinforcement‑Learned Retrieval Policy

Search‑R1 goes a step further: the retrieval strategy itself is trained via RL. A R1‑style reasoning model gains the ability to emit <search>query</search> calls mid‑reasoning; retrieved documents are inserted but masked from policy loss, and outcome‑based rewards (veRL + RAGEN) shape when to search, what to search, and when to stop.

On seven QA datasets (NQ, TriviaQA, PopQA, HotpotQA, 2WikiMultiHopQA, Musique, Bamboogle) with Qwen2.5‑7B:

Search‑R1 average EM: 0.431 vs RAG baseline 0.304.

Relative improvement: 24% (3B model: 20%).

Reference points: SFT 0.207, retrieval‑free R1 0.276, rejection sampling 0.348.

The architectural implication is larger than the scores: once retrieval is a tool call, it becomes a learnable policy . You can apply the same RL machinery that produces reasoning models to train better retrievers — impossible with a frozen embedding model.

Anthropic's Multi‑Agent Research System

The strongest non‑coding evidence comes from Anthropic's own engineering team. Their Research feature uses an orchestrator‑worker architecture: a lead agent (Claude Opus 4) decomposes the query, generates a plan, and dispatches 3–5 subagents in parallel; each subagent (Claude Sonnet 4) runs its own agentic retrieval loop, calling 3+ tools per turn; subagents return only distilled conclusions, with full tool traces isolated inside the subagent.

Result: this multi‑agent system beats a single Claude Opus 4 by 90.2% on internal research evaluations , cutting complex query research time by up to 90%, at a cost of ~15× tokens. Anthropic notes that token usage itself explains ~80% of performance variance — when task value justifies the spend, throwing more tokens at agentic retrieval yields near‑linear quality gains. Important caveat: multi‑agent helps breadth‑first tasks (research, "find all X across sources") but hurts deep serial tasks like coding.

06 Five Architectural Variants of Agentic Retrieval

By 2026 the pattern has split into at least five variants, each with representative products:

Pure Agentic (Claude Code, Devin)

No persistent index; only Glob, Grep, Read, Bash, Explore subagent. The bet: on a codebase that changes every commit, an LLM driving ripgrep in a loop beats any frozen embedding model.

Hybrid Lexical + Semantic (Cursor, Sourcegraph Amp)

Cursor documents both modes side‑by‑side: exact symbols via Instant Grep, conceptual queries via semantic search, with the agent choosing by query shape. Cursor cites internal research showing semantic + grep yields +12.5% precision . Sourcegraph's Amp layers the same agent on top of its long‑maintained code graph. Hybrid is the convergence direction for most enterprise tools.

Structural / AST‑Aware (Cline, Probe, ast‑grep)

Pure grep is lexical; pure embeddings are semantic; a third path is structural retrieval. Tools like ast‑grep and Probe use tree‑sitter to parse code, letting agents search by syntactic patterns rather than strings — e.g., "find every fetch().then(...) and rewrite to await ".

Cline's open‑source implementation is the cleanest production example: a three‑layer retrieval stack — (1) ripgrep content search with output caps, (2) fzf fuzzy file/directory search with custom scoring, (3) tree‑sitter AST extraction for multi‑language definition discovery. The agent orchestrates all three in a plan‑and‑act loop, weighting the currently open file higher. Cline reports this keeps per‑turn retrieval token usage at ~17.5% while preserving structural awareness.

Dedicated Retrieval Models (Windsurf SWE‑grep, Chroma Context‑1)

Train a small model specialized for retrieval. Windsurf's Wave 13 (early 2026) released SWE‑grep and SWE‑grep‑mini: 8 parallel tool calls per turn, 4 turns, 10× faster than generic agentic retrieval. Chroma's Context‑1 is a 20B agentic retrieval model, ~10× faster inference and ~25× cheaper on the same multi‑hop tasks.

RL‑Trained Retrieval Policies (Search‑R1, CoSearch, Agentic‑RAG‑R1)

As described above: the agent learns when to retrieve, what to retrieve, via RL, outperforming prompted agents by double‑digit margins. This is the cleanest theoretical justification for agent‑as‑retriever long‑term: it is a learnable system , not a fixed pipeline.

All five variants share one architectural premise: the agent owns retrieval . The differences lie only in what backs the tools and how those tools are created.

07 Tool Design Principles and MCP

Agent‑as‑retriever lives or dies by tool design. Anthropic's context engineering blog states the principles bluntly:

Tools must be self‑contained, fault‑tolerant, and have a crystal‑clear purpose .

Input parameters must be descriptive, unambiguous, and play to the model's native strengths .

Avoid bloated overlapping tool sets; each tool does one clear thing.

Ultimate test: if a human engineer cannot clearly say which tool fits a scenario, don't expect an AI agent to do better .

That is why Claude Code's tool surface is so small. Glob does one thing, Grep does one thing, Read does one thing, Bash handles the rest (with explicit permission gates). The model never has to choose among find_file_by_name, search_file_by_path, locate_file — each problem shape maps to exactly one tool.

If agent‑as‑retriever is the pattern, the Model Context Protocol (MCP) turns it from a Claude Code feature into an ecosystem default. Introduced by Anthropic in November 2024, MCP is a JSON‑RPC 2.0 protocol that lets any LLM‑powered host (Claude Code, Cursor, VS Code, Claude Desktop) connect to arbitrary MCP servers — programs that expose tools, resources, and prompts to the host.

The official MCP filesystem server is the cleanest production example: it exposes a carefully curated toolset — read_file, write_file, list_directory, search_files, get_file_info — under an explicit allow‑list of directories. The agent decides which to call; the protocol does not.

The implication: any MCP‑aware host becomes an agent‑as‑retriever system for anything that has a filesystem shape . Sentry exposes incidents, Postgres exposes tables, the filesystem server exposes repositories — the agent treats them uniformly: discover, search, read, refine. Once retrieval is a tool call, every data source becomes a candidate retriever without anyone building a vector index for it.

08 When Not to Throw Away the Vector Database

This pattern is not a free lunch. Several criticisms deserve serious attention:

Token Cost

Anthropic's own data is candid: the multi‑agent research system uses 15× more tokens than chat. Milvus published a critique titled "Why I'm Against Claude Code's Grep‑Only Retrieval" arguing iterative grep loops cost far more per query than pre‑computed lookup. Industry estimates: 5–30× more tokens per task than chat; complex agent loops cost $0.02–$0.10 per query vs pennies for standard RAG. Prompt caching and tool lazy‑loading recover a large chunk, but not all.

Latency

5–10 tool calls per query means seconds, not milliseconds. Fine for interactive coding; unacceptable for sub‑second user‑facing chat. SWE‑grep and Context‑1 exist precisely to compress this latency.

Massive Corpora

Grepping a 10‑million‑file monorepo isn't free. ripgrep and parallel traversal help; Explore subagents can fan out; but at petabyte scale pre‑computed indexes still win. The hybrid answer: run agentic retrieval inside a smaller, agent‑selected slice.

True Semantic Queries

"What does this codebase say about retry strategies?" is harder for grep than embeddings because the answer may be scattered across files that never use the word "retry" (using backoff, requeue, circuit_breaker instead). Agent‑as‑retriever answers by issuing multiple queries and synthesizing; Probe and AST tools answer by understanding structure; Cursor's hybrid path keeps a light semantic layer for synonym handling.

Tightly Coupled Tasks

Anthropic's own caveat: multi‑agent helps breadth‑first problems but harms deep serial tasks like coding. The shape of the work matters.

Determinism & Caching

Vector lookup is deterministic, cheap, and cacheable; agent loops are not. Production teams are converging on RAGAS, BenchmarkQED, SWE‑bench Verified for evaluation, but regression testing and SLA enforcement are harder than with static retrievers.

09 The 2026 Default: Tools First, Indexes Later

Summarized as a decision table for production teams in 2026:

Private repo code → agent‑as‑retriever (pure or hybrid). Claude Code, Devin, Cursor converge here.

Large enterprise monorepo + cross‑service queries → hybrid (Cursor, Sourcegraph Amp) or structural (Probe).

Continuously changing corpora (logs, dashboards, CRM, tickets) → via MCP server, agent‑as‑retriever; freshness beats peak recall.

Stable knowledge bases (product docs, FAQs, glossaries) → vector RAG with reranker; don't over‑engineer.

Long, layout‑heavy PDFs (financial reports, papers, contracts) → ColPali/ColQwen2 visual retrieval, or agent‑as‑retriever with pdfgrep; both work.

Whole‑corpus thematic / global questions → LazyGraphRAG. Agent‑as‑retriever struggles when no single query can hit the answer.

Multi‑hop reasoning + dynamic retrieval → Search‑R1 style RL retrieval policies, or agentic RAG frameworks.

Latency‑sensitive chat → dedicated retrieval models (SWE‑grep, Context‑1) or hybrid with vector fallback.

Breadth‑first research → multi‑agent + agentic retrieval; 90.2% gain, 15× tokens; choose when task value > token cost.

Strong serial tasks (end‑to‑end feature implementation) → single agent + strong context engineering; don't pay multi‑agent overhead.

Strict data residency / compliance → agent‑as‑retriever structurally easier to approve; no external index, no embeddings leaving the host.

For three years the dominant RAG assumption was: retrieval is an upstream system for the LLM — chunk, embed, top‑k, prompt. Agent‑as‑retriever and its formalization as JIT loading flip it: retrieval is the LLM's behavior , expressed through tool calls. The agent decides what to find, when to search again, when to stop, and how to combine what it found. The retriever is simply the most appropriate shell command, MCP server, AST query, or RL‑trained retrieval policy for the moment.

The evidence is not speculative. Claude Code shipped it and saw it beat RAG; Anthropic's multi‑agent system used the same pattern and scored +90.2%; Cursor responded by hiring the engineer who built it; Amazon achieved 94.5% of vector RAG's faithfulness with zero vector database; Search‑R1 used RL to train the retrieval policy and surpassed RAG by 24%. This pattern is now being replicated, benchmarked, productized, standardized via MCP, and trained end‑to‑end.

None of this means vector search is dead. It means vector search is no longer the default. The 2026 default is: give the agent tools, design the tools well, let it retrieve just‑in‑time ; only add vector back on workloads that genuinely need semantic generalization, massive stable corpora, or sub‑second chat. Teams that see this clearly and start running leaner stacks will benefit first; those that don't will spend 2026 maintaining vector indexes for problems that never needed them.

References

Anthropic — Effective context engineering for AI agents — anthropic.com

Anthropic — How we built our multi-agent research system — anthropic.com

Latent Space Podcast — Claude Code: Anthropic's Agent in Your Terminal (Boris Cherny & Cat Wu, 2025‑05) — latent.space

Subramanian et al. — Keyword Search Is All You Need (AAAI 2026) — arxiv.org/abs/2602.23368

Jin et al. — Search‑R1: Training LLMs to Reason and Leverage Search Engines with RL — arxiv.org/abs/2503.09516

Dive into Claude Code (arXiv:2604.14228) — arxiv.org

Model Context Protocol specification — modelcontextprotocol.io

Milvus — Why I'm Against Claude Code's Grep‑Only Retrieval — milvus.io