Mastering RAG Interview Questions: A Complete Retrieval Optimization Blueprint

Full RAG Retrieval Pipeline Overview

The RAG retrieval process consists of four stages: query understanding, coarse retrieval, fine reranking, and context organization. Basic vector search only covers one coarse retrieval method, leaving ample optimization opportunities at each stage.

Query Rewriting: Making the Retriever Understand User Intent

Short, colloquial queries often miss the terminology used in documents. Query rewriting transforms user input into a standardized form suitable for retrieval. Three common strategies are:

Synonym Expansion : Extend terms (e.g., "child" → "child, minor, teenager") via a domain synonym dictionary or LLM generation.

Query Expansion : Lengthen short queries to match document chunk size, using LLMs to generate detailed versions.

Intent‑Based Rewriting : For negation or nuanced intent, LLMs reinterpret the query (e.g., "Is nuclear radiation covered?" becomes "Is nuclear radiation excluded from liability?").

Because rewriting can introduce noise, production systems often keep both original and rewritten queries and fuse their results.

Hybrid Retrieval: Combining Keywords and Semantics

Pure vector search misses exact keyword matches, while keyword search lacks semantic understanding. Hybrid retrieval uses both vector similarity and BM25 keyword scoring. The key challenge is score fusion; a robust method is Reciprocal Rank Fusion (RRF): RRF_score = Σ 1 / (k + rank_i) where k is a smoothing parameter (commonly 60) and rank_i is the document's rank from each retrieval path.

Reranking with Deep Interaction Models

After hybrid retrieval returns 50‑100 candidates, a Cross‑Encoder reranker evaluates each [query, document] pair with full cross‑attention, yielding finer relevance scores. Compared to a Bi‑Encoder (independent encoding + dot product), the Cross‑Encoder improves Mean Reciprocal Rank (MRR) from ~0.85 to ~0.92 but is slower, so it is used only in the fine‑ranking stage.

Hard Negative Mining for Domain‑Specific Embeddings

General‑purpose embeddings (e.g., text‑embedding‑ada‑002, bge‑large) struggle with fine‑grained domain distinctions. Hard negative mining selects “appearing‑relevant but actually irrelevant” documents (rank 2‑10 from the current model) as negatives, forming (query, positive, negative) triples for contrastive learning, which sharpens the model’s ability to separate similar‑sounding terms.

Chunking Strategies and Context Compression

Fixed‑size chunking can split semantic units. Optimizations include:

Semantic Chunking : Split by logical units rather than token count.

Overlapping Splits : Add 128‑token overlap between chunks, boosting recall by 5‑10%.

Multi‑Granularity Indexing : Maintain both coarse (document/section) and fine (paragraph/sentence) indexes.

After reranking, context compression extracts the most relevant sentences or uses a small LLM to summarize chunks, raising answer accuracy from ~78% to ~85%.

Dynamic Routing: Adapting Retrieval Strategy to Query Type

Instead of static weightings (e.g., 0.5/0.5), dynamic routing first classifies the query as precise or semantic, then adjusts the vector vs. keyword balance. Simple rules (query length < 5 tokens + domain terms) or LLM‑based classifiers can drive this decision, improving MRR from ~0.82 to ~0.87.

Interview Answer Framework

When asked about RAG optimization in an interview, structure the response in four layers:

Overall Pipeline : Mention the four stages and why each matters.

Stage‑by‑Stage Techniques : Detail query rewriting, hybrid retrieval with RRF, dynamic routing, Cross‑Encoder reranking, hard negative mining, and context compression.

Trade‑offs : Discuss speed vs. accuracy (Cross‑Encoder), data requirements (hard negative mining), and potential information loss (compression).

Quantified Impact : Cite concrete improvements (e.g., recall 0.75→0.88, MRR 0.85→0.92, accuracy 78%→85%).

Be prepared to explain RRF formula, Bi‑Encoder vs. Cross‑Encoder differences, and how hard negatives are selected.

Conclusion

Improving RAG retrieval quality requires systematic enhancements across the entire pipeline. Query rewriting aligns user intent with document terminology, hybrid retrieval leverages both semantics and exact matches, reranking refines relevance with deep interaction models, hard negative mining tailors embeddings to the domain, smarter chunking preserves semantic units, context compression fits limited LLM windows, and dynamic routing selects the optimal strategy per query. Demonstrating this holistic understanding signals real‑world RAG expertise in interviews.