How Cursor Retrieves a Specific Function from a 50,000‑File Repo in Under a Second

1. A Counter‑Intuitive Observation

In a monorepo of 50 000 files, asking Cursor “where do we handle authentication?” returns the correct file in under a second. The speed is not due to a larger model but to a dedicated retrieval infrastructure: two parallel indexes, a custom‑trained embedding model, a Merkle tree, and a permission scheme that shares indexes without exposing code.

2. Why a Single Vector Index Is Insufficient

Pure semantic search (code chunk → embedding → nearest‑neighbor) can answer “where is the payment‑retry logic?” but cannot handle exact literal pattern queries such as db.execute. Regular‑expression search solves the latter, yet tools like ripgrep become seconds‑slow on large monorepos, which is unacceptable for an Agent that may need dozens of searches per task.

Technical insight: Retrieval systems must support both “semantic similarity” and “exact literal” matching; forcing one method to cover both leads to poor performance.

3. Dual‑Index Architecture Overview

The system runs a semantic index and a regex index in parallel. Results are written to temporary files; the Agent decides how much of the file to read.

4. Semantic Index: From Code to Vectors in Four Steps

4.1 Chunking by Syntax Boundaries

Instead of fixed‑size character chunks, Cursor parses source files with Tree‑sitter into an AST and creates chunks at function, class, or method boundaries. Small nodes are merged until a token limit (≈500) is reached.

def build_chunks(ast_root, max_tokens=500):
    chunks, buffer = [], []
    for node in ast_root.top_level_nodes():
        if node.kind in ("function", "class", "method"):
            if buffer:
                chunks.append(merge(buffer))
                buffer = []
            chunks.append(node.source_text())
        else:
            buffer.append(node.source_text())
            if token_count(buffer) >= max_tokens:
                chunks.append(merge(buffer))
                buffer = []
    if buffer:
        chunks.append(merge(buffer))
    return chunks

4.2 Training a Retrieval‑Specific Embedding Model

Cursor does not use off‑the‑shelf text embeddings. It trains a model on the Agent’s search trajectories: files that the Agent repeatedly opens are treated as positive relevance signals. An LLM scores the trajectories, aligning the embedding similarity with actual usefulness—a contrast‑learning approach similar to RLHF but applied to retrieval.

Technical insight: Good retrieval learns “usefulness” rather than mere “similarity”.

4.3 Storing Vectors in Turbopuffer Namespaces

Each codebase gets its own namespace in Turbopuffer, a serverless vector store backed by object storage. Hot namespaces stay in memory/NVMe; cold ones are lazily “pre‑warmed”. This design supports >1 trillion vectors and 80 million namespaces while cutting cost ~20× compared with a manually bound server setup.

5. Merkle Tree for Incremental Updates

Cursor uses a SHA‑256 Merkle tree (like Git’s object model) to detect changed files. Only subtrees with changed hashes are re‑embedded, reducing sync traffic from ~3 MB per full scan to a few kilobytes per edit.

6. Team Index Reuse + Hash‑Based Permission

When a new user opens a repo, Cursor computes its Merkle tree, derives a simhash, and looks for a similar existing namespace. If similarity exceeds a threshold, it clones the existing index via copy_from_namespace, cutting initial query latency from 7.87 s to 525 ms (median) and from >4 h to 21 s (99th percentile).

Permission is enforced cryptographically: the client must present matching Merkle hashes for each file path; the server stores only vectors and obfuscated paths, never the raw source.

Technical insight: Multi‑tenant sharing can rely on cryptographic proof of ownership instead of traditional ACLs.

7. Regex Index: Trigram as a Starting Point

The regex side uses a classic trigram inverted index (originating from Zobel et al., 1993 and popularized by Russ Cox, 2012). To improve scalability, Cursor adopts sparse n‑grams and attaches a small Bloom filter to each posting, achieving near‑quadgram precision while keeping index size manageable.

Regex indexes are built locally on the current Git commit, allowing instant updates for freshly written code.

8. Retrieval Results Are Written to Disk First

Instead of injecting large results directly into the Agent’s prompt, Cursor writes them to temporary files and returns file paths. The Agent then decides whether to read the whole file, the tail, or re‑grep it. In MCP tool calls this reduced token consumption by 46.9 %.

Technical insight: Let the Agent control context ingestion rather than the retrieval side forcing it.

9. Trade‑offs and Boundaries

Accuracy vs Freshness : Semantic index tolerates asynchronous updates; regex index requires real‑time freshness.

Performance vs Index Size : Sparse n‑grams shrink index size at the cost of extra Bloom‑filter maintenance.

Reuse Efficiency vs Preconditions : Team‑wide index reuse depends on high code similarity (≈92 %); divergent projects see less benefit.

Security vs Server Capability : Merkle‑proof permission hides source from the server but also prevents server‑side fine‑grained policy.

10. Conclusion

The system combines existing techniques—Merkle trees, trigram indexes, object‑addressable storage—with novel engineering choices: dual indexes, custom embedding trained on Agent trajectories, and a “file‑as‑context” model. These combinations enable sub‑second retrieval in a 50 k‑file monorepo.

References

Securely indexing large codebases

Improving agent performance with semantic search

Fast regex search: indexing text for agent tools

Dynamic context discovery

Cursor scales code retrieval to 1T+ vectors with Turbopuffer

Regular Expression Matching with a Trigram Index