How We Boosted RAG Recall by 15% with Practical Query Optimization Techniques

Recall@K is a key metric for evaluating Retrieval‑Augmented Generation (RAG) systems. While many try to improve it by swapping embeddings, adjusting RRF/Rerank, or tuning Milvus index parameters, the author argues that optimizing the user’s query formulation often yields more impact.

To validate the impact, an offline test set of over 100 real user queries was created and manually labeled with relevant document chunks. Recall@K (with K set to the online retrieval count, e.g., 5 or 10) was measured before and after each optimization, confirming a 15% improvement.

01 Query Rewrite

Transform colloquial user expressions into formal titles that match document headings. Example: "登录不了咋整" is rewritten by a lightweight model to "用户登录失败的常见原因与排查步骤" before searching the knowledge base, dramatically increasing hits.

Prompt used:

你是检索 query 优化器，把用户问题改写成适合向量检索的标准表述，不要回答，只输出改写后的 query。

For simple cases, rule‑based or synonym dictionaries suffice; for complex semantics, a model‑based rewrite is preferred, often using a hybrid rule + dictionary + model approach.

02 HyDE (Hypothetical Document Embedding)

When a query is too short, a lightweight model first generates a plausible answer, then uses that answer as a pseudo‑document to retrieve relevant chunks. This helps especially for 2‑3 word queries, justifying the extra model call.

03 Multi‑Query

For poorly performing queries, generate multiple parallel queries from different angles and merge results. Implementation can use rule templates or lightweight models, though the latter incurs higher cost and latency.

Prompt example:

你是 RAG Multi Query 生成器。
请从不同检索角度为用户问题生成 %d 个查询。
要求：
1. 每个查询关注一个不同角度。
2. 每行只输出一个查询。
3. 不要编造，不要解释。

Note: Multi‑Query improves recall but increases retrieval cost and noise, so it should be limited to core business scenarios.

04 Complex Query Splitting

Break a complex query into several sub‑queries, each targeting a specific aspect, then fuse and re‑rank the results. This splitting must be performed by a lightweight model, as static rules struggle with variability.

05 Contextual Completion

In multi‑turn dialogues, fill missing referents in the current user question using a lightweight model. Prompt example:

你是一个 Query 改写助手。
请根据最近对话上下文，将用户当前问题改写成一个适合知识库检索的完整问题。
要求：
1. 只补全当前问题中缺失的指代信息。
2. 不要加入无关历史。
3. 不要回答问题，只输出改写后的 Query。
4. 如果当前问题本身已经完整，直接原样输出。

06 Keyword Enhancement

For queries where semantic search alone is insufficient (e.g., error codes like ERR_5023), extract the critical keyword and perform exact keyword retrieval using Elasticsearch, complementing Milvus’s vector search. The combined results are merged, deduplicated, and then refined with RRF coarse fusion followed by Rerank.

07 Production Combo

In real projects, no single technique solves all query types. Depending on the query characteristics, choose the appropriate method: simple colloquial queries use Rewrite; queries with error codes or identifiers benefit from Keyword Enhancement; very short queries may use HyDE or Query Expansion; complex multi‑turn or multi‑aspect queries require splitting, contextual completion, or Multi‑Query. A preliminary query classification step helps select the optimal combination.