RRF in RAG: How Reciprocal Rank Fusion Solves ES-Milvus Score Fusion

In RAG systems, two retrieval paths often run in parallel: Elasticsearch (ES) for sparse BM25 retrieval and Milvus for dense vector retrieval. Their scoring logic is independent — score ranges, distributions, and meanings differ — so raw scores cannot be directly added or compared. Direct weighted fusion would let one path dominate, distorting results. Reciprocal Rank Fusion (RRF) solves this by using only rank positions, not raw scores.

1. RRF Basic Definition

RRF (Reciprocal Rank Fusion) is a multi‑retrieval coarse‑ranking fusion algorithm that depends solely on rank order, completely ignoring original retrieval scores. The base formula:

d

: a single chunk or document rank(d,r): rank of document d in retrieval path r (1‑based) k: smoothing constant, default 60 (used by both ES and Milvus built‑in RRF) r ∈ R: all retrieval paths (here two: ES and Milvus)

Weighted RRF adds a per‑path weight w_i (summing to 1) multiplied by each reciprocal rank term.

2. Four Fixed Characteristics

Only rank matters, not raw scores. BM25 scores and vector similarities have different scales; no normalization needed — cross‑engine, cross‑modal fusion works. k is a smoothing term. Denominator = k + rank; larger rank → larger denominator → smaller contribution. Larger k flattens differences between ranks; smaller k amplifies top‑rank advantage.

If a chunk is absent from a path’s recall list, that path contributes 0.

RRF is a coarse‑ranking fusion; it uses only order, not score confidence. High‑precision RAG pipelines typically add a CrossEncoder re‑ranker after RRF.

3. Plain‑Language Core Idea

RRF unifies multiple retrieval paths through rank positions:

Discard original scores, keep only each chunk’s rank within each path.

Use rank as a common yardstick to fuse multi‑path data.

Rank represents a document’s credibility in that path — higher rank, higher credibility. RRF aligns results in “order space” rather than normalizing scores to [0,1]. A chunk ranked high in both paths gets a higher summed score, reflecting consensus priority.

4. Illustrative Example: Chunk A and Chunk B

Fixed example throughout:

Chunk A: ES rank 1, Milvus rank 2

Chunk B: ES rank 2, Milvus rank 1

Ranks start at 1. Denominator = rank + 60 → rank 1 → 61, rank 2 → 62.

5. Standard RRF (Unweighted, Equal Weight)

Reciprocal values: 1/61 ≈ 0.01639, 1/62 ≈ 0.01613. Difference ≈ 0.00026 (gain per rank step).

5.1 Chunk A Total

ES: 0.01639; Milvus: 0.01613 → Sum = 0.03252

5.2 Chunk B Total

ES: 0.01613; Milvus: 0.01639 → Sum = 0.03252

5.3 Result: Tie

Both chunks score 0.03252. ES prefers A, Milvus prefers B; equal weights produce a tie. This exposes the core question: which path do we trust more? Standard RRF cannot answer; Weighted RRF addresses it.

6. Why the Tie? Commutative Property of Addition

Chunk A’s components: 1/61 + 1/62; Chunk B’s: 1/62 + 1/61. Addition commutes, so sums are identical. This generalizes: any two chunks with the same rank multiset (e.g., {3,5} vs {5,3}) will tie. RRF natively measures consensus, not path authority.

7. Weighted RRF: Assigning Different Trust Weights

Each path’s reciprocal term is multiplied by its weight before summation. Denominator, rank, reciprocal logic unchanged. Example weights: ES = 0.2, Milvus = 0.8 (trust Milvus more).

7.1 Chunk A Weighted Score

ES: 0.01639 × 0.2 = 0.00328; Milvus: 0.01613 × 0.8 = 0.01290 → Total = 0.01618

7.2 Chunk B Weighted Score

ES: 0.01613 × 0.2 = 0.00323; Milvus: 0.01639 × 0.8 = 0.01311 → Total = 0.01634

Chunk B wins. Weight direction decides winner; weight magnitude decides margin.

8. Smoothing Constant k : Controlling Rank Score Scale

k

controls how much score gain a one‑rank improvement yields. k=60: 1/61 ≈ 0.01639, 1/62 ≈ 0.01613 → diff 0.00026 k=10: 1/11 ≈ 0.0909, 1/12 ≈ 0.0833 → diff 0.00758 (28.7× larger)

Smaller

k</sub> amplifies top‑rank advantage; larger <code>k</sub> flattens curve. <code>k=60

comes from the original paper’s TREC experiments; ES and Milvus default to 60. Changing k does not alter ranking order, only widens or narrows score gaps.

9. Static Weight Configuration for ES‑Milvus Fusion

Static weights are fixed per retrieval entry point, not varying per query/chunk. They assume stable query distribution and long‑term path reliability.

9.1 Configuration Principles

Weights express trust in rank evidence, not raw score scaling. Multiply weights on reciprocal terms, not on BM25 or cosine similarity.

Sum of weights = 1 for interpretability.

Bind weights to specific entry points (FAQ, contract search, open‑ended doc search), not a single global set.

Validate offline with Golden benchmark sets via A/B testing before gradual rollout.

9.2 Three Standard Static Baselines

Keyword‑Priority (ES 0.8, Milvus 0.2) : Entity lookup, certificates, FAQ, fixed‑policy tickets. Risk: suppresses semantic matches; widen Milvus recall threshold.

Semantic‑Priority (ES 0.2, Milvus 0.8) : Long documents, papers, OCR, open‑ended questions. Risk: semantic hallucinations; require CrossEncoder re‑ranking.

Balanced (ES 0.5, Milvus 0.5) — standard unweighted RRF: Enterprise doc plaza, material library, multi‑hop agent sub‑queries. Risk: many ties; define tie‑breaker (e.g., Milvus rank).

9.3 Iteration and Expiry Signals

Static weights expire when Golden set metrics diverge (entity queries drop, semantic queries fail) or online feedback polarizes. Iterate via grid A/B on frozen Golden subsets (RAGAS flywheel), then small‑traffic canary.

10. Dynamic Weights via Intent Classification & Slot Extraction

Dynamic weights compute a distinct weight set per query.

10.1 Driving Signals

Slot signals : Detect structured identifiers (IDs, registration numbers, SKUs, contract codes). Presence → boost ES weight.

Intent signals : Classify query as entity lookup, FAQ, open‑ended, or multi‑hop. Entity/FAQ → ES; open‑ended → Milvus; ambiguous → balanced.

Slots are structured extractions, not simple keyword hits. Intent distinguishes user goal, avoiding mis‑weighting when an entity term appears in a semantic question.

10.2 Mapping Rules & Fallback

Intent = precise entity + slot hit → ES 0.8, Milvus 0.2

Intent = open‑ended semantic, no strong slots, long colloquial → ES 0.2, Milvus 0.8

Intent = multi‑hop, mixed, low slot confidence → ES 0.5, Milvus 0.5

Fallback: if intent confidence below threshold or slots conflict → force balanced weights.

10.3 Evaluation, Regression & Risk Constraints

Deploy static weights first, accumulate labeled samples, then enable dynamic branch.

Dynamic is an enhancement, not a replacement; keep a kill‑switch to revert to static entry‑point weights.

Evaluate per intent subset (entity, semantic, mixed) on Golden sets using ContextRecall and Faithfulness. Dynamic must not regress entity recall while improving semantic recall.

Risk: intent/slot errors (e.g., misclassifying open‑ended as entity) cause systematic recall drops. Log every query’s intent, slots, weights, and recall metrics; continuously retrain intent/slot models.