Challenges and Optimization Techniques for Retrieval‑Augmented Generation (RAG)

When deploying large language models (LLMs) in practice, several problems arise: lack of vertical domain knowledge, hallucinations and high entry barriers, and repeated isolated development that leads to low ROI.

From an application perspective, a solution is needed that can fill the vertical knowledge gap, lower the usage threshold, and exploit the scale advantages of LLMs. Retrieval‑Augmented Generation (RAG) is a relatively effective approach that combines a retrieval system with a generative model.

RAG Optimization Overview

Implementing a basic RAG system is straightforward, but achieving high performance requires substantial engineering effort. Below is a summary of common optimization methods and representative papers.

RAPTOR (Recursive Abstractive Processing for Tree‑Organized Retrieval)

Paper: https://arxiv.org/pdf/2401.18059

RAPTOR builds a bottom‑up tree of text chunks by recursively embedding, clustering (using GMM), and summarizing. Retrieval traverses the tree either by selecting the most similar node at each level or by flattening the tree and selecting top‑k nodes.

Key steps:

Text chunking

Embedding with SBERT and clustering with GMM

Generating summaries for clusters and re‑embedding until the hierarchy stabilizes

Experiments show RAPTOR + GPT‑4 improves accuracy on the QuALITY benchmark by 20%.

Self‑RAG (Self‑Reflective Retrieval‑Augmented Generation)

Paper: https://arxiv.org/pdf/2310.11511

Self‑RAG lets the language model decide whether to retrieve and evaluates relevance, support, and usefulness of retrieved passages before ranking them.

Key workflow:

x – input question D – retrieved documents y – generated response

If Retrieve == Yes :

Retrieve relevant passages

Predict relevance ( ISREL )

Predict support ( ISSUP ) and usefulness ( ISUSE )

Rank passages based on these scores

If Retrieve == No :

Generate next paragraph directly

Predict usefulness of the generated paragraph

CRAG (Corrective Retrieval‑Augmented Generation)

Paper: https://arxiv.org/pdf/2401.15884

CRAG introduces a lightweight retrieval evaluator that scores each (question, document) pair ( score_i ) using an evaluator E . The overall confidence determines the next action:

CORRECT – refine internal knowledge

INCORRECT – perform web search for external knowledge

AMBIGUOUS – combine internal and external sources

Generation then uses G with the input x and the processed knowledge k to produce y .

Dense X Retrieval

Paper: https://arxiv.org/pdf/2312.06648

Proposes “proposition” as a new retrieval unit (atomic factual statements). Experiments on open‑domain QA show proposition‑level retrieval outperforms paragraph‑ and sentence‑level retrieval in Recall@5 and EM@100.

Additional Techniques

• Chunking strategies (character‑level, recursive, semantic, genetic) to improve text splitting. • Query rewriting pipelines (tokenization, NER, correction, semantic expansion, HyDE, RAG‑Fusion). • Small‑to‑Big retrieval: start from fine‑grained units (sentences) and progressively expand to larger contexts. • Hybrid Retrieval: combine sparse (BM25) and dense (vector) retrieval to balance speed and semantic coverage.

Evaluation

A unified RAG evaluation framework is essential for fair comparison of models and optimizations, covering metrics such as accuracy, hallucination rate, and efficiency.

References

RAPTOR – https://arxiv.org/pdf/2401.18059

Self‑RAG – https://arxiv.org/pdf/2310.11511

CRAG – https://arxiv.org/pdf/2401.15884

Dense X Retrieval – https://arxiv.org/pdf/2312.06648

The 5 Levels Of Text Splitting For Retrieval – https://www.youtube.com/watch?v=8OJC21T2SL4&t=1933s

RAG for long‑context LLMs – https://www.youtube.com/watch?v=SsHUNfhF32s