How to Build Efficient Code Search with Vector Embeddings and AST Indexing

01 Introduction to Code Indexing

Code indexing builds a searchable catalog for large codebases, enabling fast jump‑to‑definition in IDEs, locating business logic across modules, and allowing non‑developers to understand code without a developer intermediary. Modern AI coding assistants rely on such indexes for semantic Q&A and precise context retrieval.

Common Tools and Indexing Strategies

IDE tools : pre‑built indexes support jump, autocomplete and reference analysis.

Cursor : combines traditional grep text matching with an optional vector index for semantic search.

Claude Code : uses pure grep matching, illustrating the trade‑off between simple text search and vector‑based retrieval.

Why Build an Internal Code Index?

Support AI coding scenarios : accurate code retrieval provides the context large models need, directly affecting answer quality.

Code security : a local index allows scanning and sanitising code before any external model access, preventing leakage of secrets.

Cost reduction : self‑hosted retrieval costs a few cents per query versus several dollars for third‑party AI coding services.

02 Implementation Paths: Semantic Retrieval vs. Structured Analysis

The team explored two complementary solutions: a vector‑embedding Retrieval‑Augmented Generation (RAG) pipeline and an AST‑plus‑symbol‑table static analysis pipeline.

2.1 Vector‑Embedding RAG Solution

The goal is to enable semantic code search that, together with a large language model, can answer code‑related questions.

2.1.1 RAG Core Principle

RAG follows “retrieve‑then‑generate”: before the model generates an answer, it fetches the most relevant code snippets from a knowledge base and feeds them together with the user query.

2.1.2 Chunking Strategies

Fixed‑size chunking : split code into overlapping windows of a fixed token count (e.g., 512 tokens). Simple and fast but may cut across semantic boundaries.

Semantic chunking : compute sentence embeddings and split when cosine similarity drops below a threshold, yielding higher‑quality chunks at higher computational cost.

Syntax‑aware chunking : use AST parsing to split at function or class boundaries while keeping chunk size within limits.

In the early stage, fixed‑size chunking was chosen for a good cost‑performance trade‑off.

2.1.3 Embedding

Embedding converts code tokens into high‑dimensional vectors. Semantically similar snippets (e.g., “user login verification” and userAuthentication) end up close in vector space. Each vector is stored together with metadata (original text, file path, line numbers) in a vector database.

2.1.4 Retrieval

Question embedding: the user query is encoded with the same model.

Similarity search: retrieve the top‑K (typically 5‑10) nearest vectors using cosine similarity.

Efficient indexing: employ HNSW (Hierarchical Navigable Small World) graphs for fast nearest‑neighbor lookup at million‑scale.

2.1.5 Engineering Choices

Framework : LlamaIndex was selected over LangChain for its lightweight RAG wrappers and multi‑model support.

Embedding model : internal benchmarks identified a model with 95% code‑recall on a curated test set.

LLM : DeepSeek‑V3 was chosen for its cost‑effectiveness.

Security : all code snippets pass through a sensitive‑information scanner before being sent to the LLM.

Context handling : the full recalled file is sent to the model; if it exceeds the model’s context window, the file is split and aggregated, with a default of the top‑5 files.

2.1.6 Full RAG Workflow

Indexing phase : raw code → security scan → chunking → embedding → store in vector DB.

Query phase : user question → preprocessing → question embedding → vector retrieval → top‑K code blocks → prompt construction → model generation.

2.2 AST‑Based Structured Indexing

Vector indexes excel at semantic similarity but cannot capture deep structural information such as call graphs, inheritance chains, or cross‑file impact analysis. An AST plus symbol‑table approach addresses these gaps.

2.2.1 AST Construction

Example code: let answer = 6 * 7; Lexical analysis tokenises the source into [let, answer, =, 6, *, 7, ;]. Syntax analysis then builds an AST:

Program
 └── VariableDeclaration (kind: 'let')
     └── VariableDeclarator
         ├── id: Identifier (name: 'answer')
         └── init: BinaryExpression (operator: '*')
             ├── left: NumericLiteral (value: 6)
             └── right: NumericLiteral (value: 7)

The AST provides the skeletal structure of the program, while the symbol table records identifiers, types, scopes, and definition locations. Combining both enables generation of call‑graph visualisations and impact‑range analysis (e.g., tracing method3 → method5 → method1).

03 Real‑World Applications

Intelligent code Q&A : integrated into an internal OPS platform to match front‑end APIs with back‑end services, auto‑generate business flow diagrams, and serve as a low‑cost alternative to commercial tools.

TraceAI issue localisation : a large‑model agent consumes trace logs and retrieved code snippets to autonomously pinpoint runtime problems.

Automated test‑case generation : AST‑driven change‑impact analysis suggests test cases for modified interfaces, improving coverage and efficiency.

04 Lessons Learned and Industry Comparison

Data quality is the ceiling : “Garbage In, Garbage Out” applies; high‑quality indexes are prerequisite for reliable AI assistance.

Embrace model nondeterminism : identical queries may yield different answers; engineering mitigations include multiple runs and result aggregation.

Decompose complex tasks : break large problems into smaller sub‑problems that the model can solve reliably.

Industry tools compared:

Cursor : RAG with Merkle‑Tree incremental updates and semantic chunking; high retrieval precision.

Claude Code : pure grep matching; simple but incurs higher token usage and context redundancy.

Aider : early repo‑map approach; limited scalability for large repositories.

05 Future Outlook

Cross‑repo analysis : extend indexing beyond single repositories to support micro‑service architectures.

Issue‑localisation agents : embed TraceAI capabilities into daily dev‑ops to reduce mean time to repair.

Automated documentation : generate and keep API docs and business flowcharts up‑to‑date from code and AST analysis.

Combining the semantic strengths of RAG with the precise structural insights of static analysis promises a more comprehensive code‑understanding platform, ultimately saving developers time in navigation and logic comprehension.