Vector Databases Aren't Dead: How Claude Code & Cursor Are Redefining RAG for Agents

Introduction: The Gap Between RAG Demos and Production

Teams building RAG systems often see a gap: demos work with chunking, embeddings, and top-k retrieval, but production reveals stale indexes, semantic search returning irrelevant code, and answers citing outdated documents that no one trusts for critical actions.

The claim "vector databases are dead, Claude Code and Cursor abandon RAG" sounds plausible. The author investigates three sources and finds they describe different conditions.

Three Sources, Three Different Conditions

Claude Code : Early versions tried off-the-shelf RAG, then switched to letting the agent use glob, grep, and other tools to find files. Boris Cherny said Agentic Search is "way ahead" based on internal usage and benchmarks, not public controlled experiments.

Cursor : In November 2025, Cursor published an experiment: adding semantic search to the agent improved offline code QA accuracy by 12.5% on average; online A/B tests showed a 2.6% increase in code retention for codebases over 1,000 files. Cursor also observed heavy grep usage, but the combination of both searches worked best.

"Keyword Search Is All You Need" paper : Compared a fixed-configuration RAG baseline against an agent that rewrites queries and calls rga and pdfgrep. Across five document QA datasets, the agent achieved 94.52% faithfulness, 88.05% context recall, and 91.48% answer correctness relative to the RAG baseline. The authors note limitations: performance drops with larger documents, multimedia is hard, context window and vague questions affect results. This shows multi-round keyword search can approach a fixed RAG baseline in some tasks, but does not prove semantic search has lost value.

Retrieval Is Moving Into the Agent Runtime Loop

The author narrows the conclusion: retrieval is becoming part of the agent's execution process. Traditional RAG prepares context once before the model call; now the agent searches, finds evidence, switches tools, and decides the next step based on new evidence.

Vector databases remain in the architecture, but their position shifts.

Single Retrieval Cannot Keep Up with Troubleshooting

Traditional RAG handles one round: question → retrieve → answer. Agents receive tasks where each search depends on the previous evidence.

Example: debugging "payment request occasionally executes twice." The agent starts with an error code, logs point to HTTP timeout, config reveals message queue retry, tracing the call chain leads to checking idempotency keys and transaction boundaries. Each discovered clue changes the next search's keywords, scope, and tools. A single upfront retrieval cannot cover the whole investigation.

SWE-bench early numbers: RAG baseline 1.96% solve rate vs. SWE-agent (with tools, code editing, test running) 12.47%. The author cautions this is not a direct retrieval comparison because model, workflow, and execution changed, but it shows software engineering tasks require acting on evidence and verifying changes.

Another issue: agents modify code they are retrieving. If search still reads an old index after a config change, analysis falls on stale code. Cursor's fast regex index overlays user and agent uncommitted changes on the Git baseline so search covers newly written content.

Retrieval is no longer a pre-model step; it is part of the troubleshooting and modification loop.

Figure 1: Fixed RAG completes retrieval upfront; agent continues searching based on new evidence

Grep and Vector Search Each Suit Different Clues

Codebases contain multiple clue types. Exact search (function names, paths, config keys, error codes) is most efficient. Semantic search finds entry points when the exact phrasing is unknown (e.g., "where is refund permission validation?"). LSP, AST, or graph indexes are better for definitions, references, and call graphs.

Claude Code and Cursor's choices differ but fit their products. Claude Code puts filesystem and generic tools on the main path, avoiding a pre-built index and its update, security, and maintenance overhead. Trade-off: runtime exploration may be slower, consume more tokens, and follow false leads.

Cursor, facing many large codebases, keeps semantic indexing and adds fast regex search. Both tools coexist; different queries take different paths.

Anthropic later framed this as Just-in-Time Context Loading: keep lightweight pointers (file paths, queries, links) and fetch content on demand. Claude Code calls its approach a "hybrid strategy": CLAUDE.md preloaded into context, specific files found via glob and grep as needed.

One leans toward runtime search, the other retains semantic indexing. No universal ordering; codebase size, update frequency, and latency budget change the choice.

Putting Vector Databases Back in the Index Layer

In database architecture, indexes and primary stores are separate: indexes speed reads, can be rebuilt; business facts stay in the owning system.

Applied to agent systems, the author prefers the same boundary: vector databases are retrieval projections that provide candidates; facts must be verified against authoritative sources.

The chain splits into five layers:

Authoritative sources → Retrieval projections → Runtime routing → Current working set → Execution & verification

Figure 2: Five-layer boundary of agent context system

Leftmost layer holds verifiable sources: current code in working tree and Git, order status in business DB, release results in artifacts and deployment receipts. Markdown saying "service deployed" doesn't prove the version is running; knowledge base saying "order refundable" doesn't prove this order qualifies.

Second layer: vector indexes, inverted indexes, regex indexes, AST, knowledge graphs, caches. They narrow scope, can update asynchronously, can be rebuilt. Conflicts with source resolved in favor of authoritative sources.

Third layer: at runtime, the agent selects tools based on clues. Function names → exact search; natural language → semantic search for entry point; real-time balance → controlled API. Harness limits permissions, result size, call count, timeout to prevent runaway exploration.

Fourth layer: retrieved content not all shoved into context. Current goal, relevant source, necessary tool results, and phase state form a working set. Large files read in segments; long logs viewed near errors; unused material stays outside. Working set adjusts as tools bring new evidence.

Fifth layer: agent modifies code, runs tests, views diffs; calls business APIs, checks receipts and current state; conclusions retain original citations. Failure yields new clues, search enters next round.

RAG isn't deleted; it returns to the retrieval projection layer.

Markdown Solves a Different Problem

Claude Code and OpenClaw use Markdown for project rules and long-term memory. Strengths: human-readable, directly editable, Git-tracked, accessible via path and text tools.

Project conventions, task notes, failure records, decision logs suit this form. They need human review, occasional rollback, and audit trails. If content hides behind an index, maintenance and accountability become harder.

Markdown is poor at real-time state and high-concurrency writes. Many files still need indexing; concurrent edits conflict; transactions and fine-grained permissions are not its forte.

OpenClaw's public design combines both: long-term memory written to Markdown; with embeddings configured, memory_search blends vector similarity and keyword matching. Files store content; indexes aid discovery.

Here Markdown and vector indexes are partners, not replacements. The former keeps readable, editable, auditable content; the latter reduces lookup cost.

Applying to Concrete Scenarios

In codebases, first ensure search covers the current working tree. Exact and symbol search for known clues; semantic search for unfamiliar entry points. If the agent can't read a just-edited file, offline recall doesn't solve the immediate problem.

For hundreds of thousands of relatively stable documents with large query-document phrasing gaps, vector recall remains valuable. Keywords, metadata filtering, and reranking补足 precision; version and permission enter retrieval conditions.

Inventory, orders, monitoring, permissions are fast-changing data with clear system owners. Letting agents query source systems via controlled tools usually avoids an extra replication layer and reduces stale reads.

Figure 3: Different clues suit different tools; verify against authoritative sources before acting

At design reviews, disagreements usually center on three questions:

Which system holds the ultimate fact, and can we fall back to it when the index expires?

Are the agent's clues exact symbols, natural language, or real-time state?

If retrieval errs, does it only cause a wrong answer, or trigger a real operation?

Answers rarely point to a single technology. The first two affect retrieval path; the last dictates verification depth.

Recall Metrics Don't Explain Task Completion

Recall@k, Precision@k, MRR, latency still matter, but they describe a single retrieval. An agent may miss the target file in round one, find it in round two via an error code; or retrieve a relevant snippet in round one but modify code based on an outdated version.

Cursor's experiment tracked not only offline QA accuracy but also code retention and user follow-ups. It placed retrieval results back into the full task, not just similarity scores.

For evaluating a custom agent, the author would care about real tasks run end-to-end: did it find current evidence, how many search rounds, could it switch tools on failure, did modifications pass tests, how much human rework remained. Putting cost and latency in the same results clarifies trade-offs.

After running, some tasks need only grep, some require vector recall, some shouldn't query a knowledge base at all.

Where I'd Place the Vector Database

Returning to the title: Claude Code didn't declare vector databases obsolete; Cursor didn't abandon semantic search. The keyword search paper shows something else: under constrained tasks and config, an agent using multi-round keyword retrieval can approach a fixed RAG baseline.

The more interesting shift: applications no longer try to prepare all context before the first model call. The agent keeps searching based on new evidence, and retrieval tools change with the task.

In architecture design, the remaining questions are concrete: where facts live, index freshness, agent toolset, context retention, post-execution verification.

In future agent architecture diagrams, I'll still keep the vector database, but not in the center. The center belongs to the continuous search-act-verify runtime loop.

References

Latent Space: Claude Code team interview (https://www.latent.space/p/claude-code)

Shreyas Subramanian et al.: Keyword Search Is All You Need (https://arxiv.org/abs/2602.23368)

SWE-bench: Original benchmark (https://www.swebench.com/original.html)

Anthropic: Effective context engineering for AI agents (https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents)

Claude Code: How Claude remembers your project (https://code.claude.com/docs/en/memory)

Cursor: Improving agent with semantic search (https://cursor.com/blog/semsearch)

Cursor: Fast regex search: indexing text for agent tools (https://cursor.com/blog/fast-regex-search)

Cursor: Securely indexing large codebases (https://cursor.com/blog/secure-codebase-indexing)

OpenClaw: Memory overview (https://docs.openclaw.ai/concepts/memory)

Code example

相关阅读：