Skip to content
RAG Deep Dive: Principles, Optimization & Production Implementation

RAG Deep Dive: Principles, Optimization & Production Implementation

This comprehensive guide explains Retrieval-Augmented Generation (RAG) from core principles to production deployment, covering offline indexing (document loading, chunking, embedding, vector storage) and online query pipelines (vector retrieval, reranking, prompt assembly, LLM generation) with concrete e-commerce and AI writing examples.

LouZai
LouZai
LouZai

01 RAG Is Open-Book Exam

LLMs have three fundamental problems: hallucination (confidently fabricating answers), knowledge cutoff (training data has a deadline), and ignorance of private data (company docs, personal data). The root cause: LLMs are closed-book test-takers — they rely on internal parametric memory and never say "I don't know." RAG solves this by making the model open-book: before answering, retrieve relevant passages from a knowledge base and inject them into the prompt so the model answers grounded in evidence.

One sentence: RAG = open-book exam. The model doesn't change; what changes is whether it can find the right page before answering.

02 Full Pipeline: Offline 4 Steps + Online 4 Steps

The complete RAG pipeline splits into two phases:

Offline: Build Index (one-time / periodic)

Document Load → Chunk Splitting → Embedding Vectorization → Store in Vector DB

Online: Query (every question)

User Question → Vector Retrieval → Rerank → Assemble Prompt → LLM Generate

2.1 Offline: Turn "Refund Policy v2.1" into Searchable Index

Scenario: E-commerce company with a 5,000-word Markdown refund policy.

Raw doc: "Refund Policy v2.1" (5,000 words Markdown)
↓ Load: parse to plain text, preserve heading hierarchy
↓ Chunk: recursive split by section — "7-day no-reason return", "refund timeline", "shipping cost" each become a chunk
Each chunk ~500 words, 10% overlap to prevent key clauses landing on split boundaries
↓ Vectorize: each chunk through embedding model → 1,536-dim vector
↓ Store: vector + original text + metadata (section path: after-sales/return/shipping) into vector DB

Output: dozens of "policy fragments" waiting for user questions

① Document Loading

Convert various formats to plain text. PDFs are hardest: scanned PDFs need OCR; tables often garble; code blocks must preserve formatting and language markers. For PDF sources, allocate at least one-third of engineering time to this step.

② Chunk Splitting

Why chunk? (1) Documents too long for context window; (2) embedding models have input limits (typically hundreds of tokens); (3) only after chunking can you precisely hit "the relevant segment" instead of dumping the whole doc.

Policy doc split by section recursively:

7-day no-reason return ~500 words — Return conditions, scope, non-applicable items

Refund timeline ~400 words — Review time, arrival time, different payment methods

Shipping cost rules ~300 words — Buyer vs seller bears cost, quality-problem exception

Special product returns ~350 words — Fresh, customized, digital product special rules

Three splitting strategies:

Fixed-size : Every N characters — Simple, predictable; ignores structure, chops paragraphs/tables

Structure-aware : Recursive by heading→paragraph→sentence — Respects document structure; slightly complex

Semantic : Cut where adjacent sentence embedding similarity jumps — Purest topic per chunk; extra similarity compute, higher cost

Chunk size: no universal value; must empirically tune. Too large (2000+ words) mixes topics, dilutes vector; too small (<100 words) loses semantic completeness. Starting heuristic: 256–1024 tokens, 10–20% overlap, then adjust per retrieval quality. Overlap prevents boundary truncation — key info falling exactly on split line won't be lost.

③ Embedding Vectorization

Embedding maps text to numeric vectors so semantically similar texts have nearby vectors. Analogy: assign coordinates to fruits — apple (sweetness 8, acidity 3, hardness 7), orange (7,6,3), potato (2,1,8). Apple and orange are close; potato is far. Embedding does this automatically with hundreds/thousands of learned dimensions.

Vectors capture semantic directions. Classic example: King - Man + Woman ≈ Queen — not arithmetic but moving along a "gender" axis in vector space.

Similarity measured by cosine similarity — direction alignment regardless of magnitude. Two articles with same content but different lengths have different vector lengths but same direction → similarity near 1.

⚠️ Critical Pitfall: Index build and query MUST use the same embedding model.

If index built with 1,536-dim model but query falls back to 384-dim local model, vector spaces are incompatible → retrieval returns garbage. Silent bug: no errors, no timeouts, just nonsense results. You'll blame chunking and waste days tuning splits. Fix: store embedding model name in metadata at index time; at query time verify match, hard-fail if mismatch — explicit failure beats silent hallucination.

④ Vector Database Selection

Chroma : Lightweight embedded, local file persistence — Personal projects, ≤100k chunks

PGVector : PostgreSQL extension — Existing PG infra, avoid extra ops

Milvus : Distributed professional vector DB — Production scale, high availability

FAISS : Local library (not a service) — Offline batch experiments

Core principle: "good enough," not "strongest" — if data <100k, Milvus's distributed capability is wasted.

Three indexing details: (1) Store original text alongside vector — retrieval returns text for prompt, not vector; (2) Store metadata (source section, category, chunk index) — later filtering depends on it; (3) Batch insert — avoid memory spikes.

2.2 Online: User Asks "Bought 3 Days Ago, Want Return, Who Pays Shipping?"

Step 1: Vector Recall (Coarse Filter)

User question → embedding → vector DB similarity search → top 20. Example results:

Rank  Section                 Summary                                                    Sim
1     7-day no-reason return  "Within 7 days of receipt..."                              0.89
2     Refund process          "1-3 business days review, refund original route..."       0.85
3     Shipping cost rules     "Return shipping borne by buyer; quality issue by seller..." 0.83
4     Special product return  "Fresh, customized not eligible for no-reason return..."     0.78
5     Return timeline         "Count 7 natural days from day after receipt..."           0.76

User asked about "shipping cost" but "shipping cost rules" ranks 3rd. Reason: vector search compares overall semantic similarity; "7-day return" and "refund process" are closer to the broad topic "return." Coarse filter recalls broadly but ranks imprecisely — normal.

Step 2: Rerank (Fine Sort)

Take top 5, feed query+doc pairs into cross-encoder for per-token scoring:

Rank  Section                 Cross-Encoder Score  Change
1     Shipping cost rules     8.72                 ↑ from 3 to 1
2     7-day no-reason return  6.35                 ↓ from 1 to 2
3     Special product return  4.18                 ↑ from 4 to 3
4     Refund process          2.05                 ↓ from 2 to 4
5     Return timeline         1.32                 ↓

"Shipping cost rules" jumps from 3 to 1. Coarse filter compares "overall likeness"; rerank compares "local match." Cross-encoder concatenates query and doc, attends token-by-token — user's "shipping cost" matches doc's "shipping cost rules" exactly → high score; "refund process" lacks "shipping cost" token → score drops.

Analogy: coarse filter = headhunter scanning résumés in 5 seconds; rerank = interviewer deep-diving each candidate. Coarse then fine = recall broad + rank precise.

Take top 3 to next step.

Step 3: Assemble Prompt

Inject top 3 original texts into prompt with source labels and anti-hallucination instruction:

You are e-commerce support assistant. Answer based on reference materials below.
If materials lack relevant info, honestly say "I cannot answer" — do not fabricate.

【Ref 1】Shipping cost rules
Return shipping borne by buyer; quality issue → seller bears.
Shipping cost deducted at refund time, no separate payment.

【Ref 2】7-day no-reason return
Within 7 days of receipt, eligible if product intact, packaging complete.
Not applicable: fresh, customized, unsealed digital products.

【Ref 3】Special product return
Fresh, customized not eligible for no-reason return.
Digital products unsealed require inspection report for return.

User question: Bought 3 days ago, want return, who pays shipping?

Source labels enable traceability; "say I don't know" constrains model.

Step 4: LLM Generate

You purchased 3 days ago, within the 7-day no-reason return window, so you can apply for return. Return shipping is borne by you, deducted at refund. If returning due to product quality issue, shipping is borne by seller.

Every number and rule traces to references: "7 days" from Ref 2, "buyer bears shipping" and "quality issue seller bears" from Ref 1. If coarse filter missed "shipping cost rules," even the smartest model would hallucinate.

One sentence: Coarse filter ensures recall breadth, rerank ensures precision ranking, prompt constrains generation. Nail these three, online RAG won't fail.

03 How to Retrieve Accurately

3.1 Retrieval Landscape

Vector (dense) : Semantic similarity — Strengths: synonyms, paraphrases, fuzzy queries; Blind spots: exact codes, proper nouns; Speed: Fast

BM25 (sparse) : Term frequency + rarity — Strengths: exact match, model numbers, error codes; Blind spots: synonyms, semantic rewrites; Speed: Fast

Hybrid : Parallel two-lane + RRF fusion — Strengths: blind spots complement; Blind spots: cost doubles; Speed: Medium

Examples: Vector search "puppy" finds "young dog" (semantic near) but misses exact "HTTP-403" (semantic near but not exact code). BM25 nails "HTTP-403" but misses "kitty" vs "cat" (no synonym understanding). Hybrid runs both, fuses via RRF.

Pipeline: Retrieval (pick one) → responsible for "recall breadth" → Rerank → responsible for "precision ranking" → Prompt assembly → responsible for "constrain generation."

3.2 Hybrid Retrieval: Two Blind Spots Complement

Vector catches semantics, BM25 catches exact keywords. Fusion challenge: vector scores 0–1, BM25 scores tens — different scales, can't weighted-sum directly. Solution: RRF (Reciprocal Rank Fusion) — ignore scores, only use ranks.

Combined score = Σ 1 / (60 + rank)  # rank = position in that lane's results

Manual calc: Doc A rank 2 in vector + rank 3 in BM25 → 1/62 + 1/63 = 0.0320. Doc B rank 1 in vector + rank 50 in BM25 → 1/61 + 1/110 = 0.0255. Doc B wins one lane but loses to Doc A strong in both. RRF logic: both lanes must vouch for you; one-trick ponies lose.

Hybrid is NOT mandatory. Decision rule: will users search exact strings? Error codes, model numbers, names → hybrid required. Pure semantic tasks → vector alone suffices.

3.3 Rerank: Precision After Coarse Filter

Hybrid solves "recall breadth" but order is still wrong. Vector recall uses approximate nearest neighbor (ANN) for speed — top 1 by ANN isn't necessarily most relevant. Like headhunter fast-screening 30 résumés; interviewer (rerank) deep-dives each.

Why swap model? Architecture difference:

Bi-Encoder (Dual Encoder) : Query and doc encoded separately to vectors, then distance; encoders never see each other; indirect vector comparison; very fast (doc vectors precomputed offline); used in recall stage (massive coarse filter)

Cross-Encoder : Query and doc concatenated, fed together; token-level cross-attention, captures fine-grained match; slow (each pair computed real-time); used in rerank stage (dozens of candidates)

Bi-encoder = headhunter filtering by "degree + years"; cross-encoder = interviewer matching JD line-by-line. Coarse then fine = broad recall + precise ranking.

In e-commerce case, coarse filter put "shipping cost rules" at #3 (semantic relevance to "return" but not most direct); cross-encoder token-match on "shipping cost" → jumps to #1.

3.4 How to Quantify Effectiveness

Three metrics, three questions:

recall@k : Did we find the right one? — Is correct answer in top k?

MRR : Is it ranked high enough? — Position of first relevant result

nDCG : Overall list quality? — Relevant at top, irrelevant at bottom

E-commerce example: user asks "who pays shipping", ground truth in "shipping cost rules" chunk.

Scheme A (vector only, no rerank):
1. 7-day return ← wrong
2. Refund process ← wrong
3. Shipping cost rules ← correct! rank 3
4. Special product return
5. Return timeline

Scheme B (vector + rerank):
1. Shipping cost rules ← correct! rank 1
2. 7-day return
3. Special product return
4. Refund process
5. Return timeline

recall@3: both 1 (correct in top 3). MRR: A = 1/3 ≈ 0.33, B = 1. MRR captures "how fast found." nDCG: B higher because correct at top, irrelevant sunk. Each metric owns a dimension: recall = "found?", MRR = "fast?", nDCG = "list quality?"

Real evaluation: sample 30–50 real user questions, human-label ground truth, change one param, run all three metrics — numbers up = real improvement. "Feels better" doesn't count.

One sentence: Hybrid fills exact-match gap, rerank promotes correct to top, RRF fuses by rank only; optimize watching recall@k — numbers up = real gain.

04 How to Retrieve Smartly

4.1 Three Techniques: Rewrite, Filter, Contextualize

Query Rewriting

User's raw query often unfit for retrieval. "How to return that thing?" — engine doesn't know "that thing." Fix: LLM rewrites to search-friendly form first.

User: "How to return that thing"
↓ LLM rewrite
Rewritten: "What is the refund/return process?"
↓ Retrieve
Hits: refund policy, refund timeline...

Cost: one extra LLM call. Gain: huge for ambiguous queries. User doesn't know your KB terminology; LLM does — let it translate.

Metadata Filtering

Filter by tags before vector search. KB has "refund policy", "loyalty points", "delivery times". User asks return question → filter category="after-sales" → vector search only in after-sales — cuts half irrelevant, precision jumps.

User: "Bought 3 days ago, want return, who pays shipping?"
↓ Metadata filter: category = "after-sales"
↓ Vector search: only in "after-sales" category
↓ Results: return-related, no "loyalty points" or "delivery times" noise

Filter may misclassify, but cost low — worst case falls back to full KB search, no wrong results introduced. Goal: cut obvious noise, not perfect classification.

Contextual Retrieval

Chunking breaks coreference. "The company's revenue grew 3%" — "the company" refers to prior paragraph, but split across chunks. Embedding such fragment loses context.

Anthropic's fix: before embedding, LLM prepends context summary to each chunk.

Raw chunk: "The company's revenue grew 3%, Q3 revenue broke 5B."
↓ LLM adds context
Contextualized chunk: "This text discusses Xiaomi Group's 2024 financial report.
The company's revenue grew 3%, Q3 revenue broke 5B."
↓ Embed
Vector now contains "Xiaomi Group" semantics

Cost: one extra LLM call at index time. Gain: long-doc recall quality. Anthropic measured: Contextual Embedding + BM25 + Rerank → retrieval failure rate down 67%.

4.2 Agentic RAG: Retrieval Is a Tool, Not Mandatory Pipeline

Traditional RAG = mindless pipeline: every question retrieves, regardless of need. But retrieval costs latency, tokens, and noise — irrelevant retrieved docs degrade generation.

Agentic RAG makes retrieval a tool the model decides to use: whether to search, what to search, how many rounds.

E-commerce scenarios:

No search needed: "What is 7-day no-reason return?" — common knowledge, model knows, answer directly. Forced search wastes time.

Single search: "Bought 3 days ago, who pays shipping?" — single-hop, one search "shipping cost rules" suffices.

Iterative search: "Double-11 purchase want return, can coupon still be used?" — involves two distinct knowledge pieces, one search incomplete.

Round 1 search:
Think: question covers both "return" and "coupon", search return policy first
Search: "Double-11 return policy"
See: "Promo-period purchases, return → coupon not refunded"
But user may also ask "how is refund amount calculated" — promo price ≠ original price

Round 2 search:
Think: need "promo order refund calculation" rule
Search: "Promo order refund calculation"
See: "Promo orders refunded at actual paid amount, coupon-deducted portion not refunded"

Synthesize both rounds:
"Double-11 items returnable, refund based on your actual paid amount.
Coupon-deducted portion not refunded, coupon not restored after return."

Traditional RAG single-shot either misses refund calc or mixes both with messy ranking. Agentic RAG: after round 1 sees gap, constructs precise round 2 query — like researcher cross-referencing until material sufficient, then writes.

One sentence: Query rewrite rescues vague questions, metadata filter cuts half noise, Contextual Retrieval restores broken context; retrieval frequency follows content reusability — common knowledge answers directly, Agentic RAG lets model decide.

05 Hands-On Implementation

Project: article-writer, an AI agent that writes WeChat articles. Key difference from e-commerce support:

Support retrieves "answers"; writing retrieves "style."

Support: user asks "who pays shipping" → KB returns "shipping cost rules" text as answer. Writing: user says "write article about RAG" → KB doesn't need RAG knowledge (LLM has it); needs "how does Louzi write technical articles" — openings, analogies, paragraph length, tone.

5.1 Offline: 197 Articles → Style Reference Library

Source: 197 Markdown articles in 11 directories. Tech-heavy: Spring 39, Java/JVM 30; Personal: Growth 29, Management/Essays 28.

Each chunk tagged with two metadata fields: style (technical/personal) and category (source directory). Writing technical article → filter style=technical; writing essay → filter style=personal. Filter style, not knowledge.

Chunking: Real Article Breakdown

Article: "Manually Build a Redis Distributed Lock" (7,745 words). Chunk params: 1,000 chars/chunk, 150 char overlap, back-scan from split point to blank line for paragraph boundary.

Produces 10 chunks. Representative samples:

Chunk 1 (988 chars): Article opening
"Hi, I'm Louzi. For Java folks, we don't need to manually build distributed lock, just use Redisson. But these wrapped components make us lazier..."

Chunk 2 (796 chars): Core lock timeout problem
"Compare with current time to judge if lock expired. If not expired, return; if expired, reset expiry, acquire lock. Any other issues? Of course! In concurrent scenarios, threads A and B simultaneously..."

Chunk 3 (907 chars): Concurrent scenario analysis
"Extending or shortening lock expiry — any issues? In real concurrency, reaching this step means 'simultaneous' arrival, time diff tiny..."

Chunk 1 = typical tech opening — "Hi, I'm Louzi" greeting, colloquial lead-in. Chunk 2 = signature sentence pattern — pose technical problem, rhetorical question pivot. Chunks 8–10 = identical boilerplate footer ("Follow public account...") — noise, should have been cleaned pre-index; this library skipped that step.

Indexing: Real Parameters

CHUNK_SIZE : 1,000 chars (~500–700 Chinese tokens) — Tried 2,000 → topic mixing severe; tried 500 → semantics too fragmented. 1,000 = empirical sweet spot

CHUNK_OVERLAP : 150 chars (~15%) — Prevent key info straddling split boundary

Paragraph boundary back-scan : From split point backward to blank line — Avoid cutting mid-paragraph

Embedding model : OpenAI text-embedding-3-small (1,536-dim) primary; fallback local bge-small-zh-v1.5 (384-dim) — Priority to quality; fallback for resilience

Batch insert : 50 chunks/batch — Avoid memory spike

Metadata : style (technical/personal) + category — Auto-inferred from directory structure, used for style filtering

197 articles → ~2,800 chunks → 28 MB vector file. Fits in single ChromaDB embedded file.

5.2 Online: Writing an RAG Article — How System Finds Style References

Step 1: Style Classification

User input: "Write technical article about RAG principles and practice." Keywords "RAG", "principles", "practice" → technical → filter style=technical → excludes personal-style chunks ("My anxiety", "35-year-old crisis").

Step 2: Vector Recall (Coarse)

Query "RAG principles practice technical article" → top 20. These chunks are NOT RAG knowledge; they are "Louzi's technical writing style samples."

Rank  Source Article                              Sim    Style Trait
1     Manually Build Redis Distributed Lock       0.842  Colloquial opening: "Hi, I'm Louzi"
2     MySQL Single Table Max Rows                 0.817  Data-driven: lead with numbers
3     Redis High Availability Principles          0.814  Life analogies for technical concepts
4     Only Know Single-Node Cron? Multi-Node YYDS 0.794  Rhetorical questions: "Any other strategies?"
5     MySQL-Redis Data Consistency                0.788  Table comparisons for solution diffs

Coarse filter noise: test output logs, "recommended reading" links — unrelated to "writing style" but keyword overlap pulls them in.

Step 3: Rerank (Fine)

Cross-encoder rescores:

Rank  Source Article                              Cross-Encoder  Change
1     Manually Build Redis Distributed Lock       0.998  Stays #1
2     MySQL Single Table Max Rows                 0.996  Stays #2
3     Redis High Availability Principles          0.994  ↑ from #3 (noise dropped, backfilled)
4     MySQL-Redis Data Consistency                0.992  ↑ from #5
5     Only Know Single-Node Cron? Multi-Node YYDS 0.990  ↓ from #4

Noise (test logs, recommendation links) falls out of top 5 — cross-encoder token-match shows low relevance to "RAG technical article." Same pattern as e-commerce: coarse = "overall likeness", fine = "local match."

Step 4: Assemble Prompt

Top 3 injected into system prompt:

You are WeChat article writing assistant. Mimic the writing style of the following historical articles.

【Style Ref 1】(Source: Manually Build Redis Distributed Lock)
"Hi, I'm Louzi. For Java folks, we don't need to manually build distributed lock, just use Redisson. But these wrapped components make us lazier."

【Style Ref 2】(Source: MySQL Single Table Max Rows)
"I said MySQL single table best not exceed 20M rows, interviewer told me go wait for notice?"

【Style Ref 3】(Source: Only Know Single-Node Cron? Multi-Node YYDS!)
"Leave a question: if one record keeps failing, every fetch gets that bad record first — what strategy pushes it later?"

Mimic the tone, sentence patterns, paragraph rhythm above to write a technical article about RAG principles and practice.

Agent imitates tone, syntax, rhythm. RAG knowledge comes from LLM; style comes from retrieved historical fragments.

Same pipeline as e-commerce support — chunk, index, coarse, fine, prompt. Different purpose, same plumbing.

Honest caveat: RAG for style matching works poorly. RAG retrieves by semantic similarity → finds "topic-similar text", not "style-similar text." Two articles on totally different topics can share identical style — RAG can't distinguish. This case demonstrates adapting one pipeline to different needs. In production, RAG excels at knowledge retrieval (policies, docs, processes) — e-commerce support is its true battlefield.

One sentence: 197 articles → 2,800 chunks → 28 MB vector DB → full query from coarse to fine — theory and artifact side by side = true RAG understanding.

06 Closing

RAG does one thing: before the model answers, fetch the right reference material for it.

Chapter 1 — Why RAG: hallucination, stale knowledge, private data blindness → open-book fix.

Chapter 2 — Full pipeline: offline 4 steps (knowledge → searchable vectors), online 4 steps (question → best answer), walked end-to-end with e-commerce example.

Chapter 3 — Retrieval accuracy: vector for semantics, BM25 for keywords, RRF fusion, rerank for precision ranking.

Chapter 4 — Smart retrieval: query rewrite for vagueness, metadata filter for noise, Contextual Retrieval for broken context, Agentic RAG for on-demand search decisions.

Chapter 5 — Real project validation: 197 articles → 2,800 chunks, complete coarse-to-fine process, theory matched with artifact.

Original Source

Signed-in readers can open the original source through BestHub's protected redirect.

Sign in to view source
Republication Notice

This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactadmin@besthub.devand we will review it promptly.

RAGVector DatabaseEmbeddingRetrieval-Augmented GenerationLLM ApplicationRerankHybrid SearchAgentic RAG
Reader feedback

How this landed with the community

Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.