Naive RAG Outperforms Complex Agentic Memory: RAW-RAG Beats 12 Systems with Simple Retrieval

Motivation: Memory Mutations Are Extremely Rare

The authors first audited how often dynamic state changes actually occur in memory tasks. They examined all write decisions made by Mem0 on the LoCoMo and LongMemEval benchmarks:

UPDATE + DELETE accounted for only 4.39% / 2.46% of decisions; effective mutations were merely 1.90% / 1.34%.

Many proposed “updates” were duplicate content or formatting errors.

They then ran a controlled ablation: replacing UPDATE with blind APPEND and DELETE with NONE. LoCoMo accuracy increased from 67.36% to 69.42% , with gains across every sub‑category. This suggests explicit state maintenance yields negligible benefit, motivating a “strong Reader reasons directly from raw evidence” approach.

Method: How Simple RAW-RAG Really Is

RAW-RAG’s pipeline has only three steps:

Chunking : Split dialogue into 256‑token windows with 32‑token overlap — no LLM extraction or summarization.

Retrieval : Cosine similarity recall of Top‑128 chunks.

Reranking : Cross‑encoder precision rerank, take Top‑15 for the Reader LLM.

No memory graphs, no atomic fact extraction, no iterative retrieval — state reconstruction is entirely delegated to the Reader at inference time. The pipeline diagram (Figure c) is a straight line, whereas EverMemOS and Memora have complex Build/Store/Recall stages.

Main Results: First Place in Six Settings

Evaluated on LoCoMo, LongMemEval, MEME, and MemoryAgentBench against 12 mainstream memory systems (Mem0, MemOS, EverMemOS, MemMachine, Memora, etc.):

LoCoMo (GPT‑4o‑mini / 4.1‑mini): 89.07 / 93.70 , leading by 0.65–0.95 points.

LongMemEval: 87.53 , leading by 0.13 points.

MEME (GPT‑4o‑mini): leading by 2.24 points.

MemoryAgentBench: leading by 4.48–5.53 points.

Only loss: MEME + DeepSeek‑V4‑Flash, where MemMachine outperformed by 1.52 points.

Efficiency: on the million‑token BEAM‑1M benchmark, RAW-RAG ingested all history in 0.3 hours vs. 27 hours for Memora — an 83× speedup .

Ablation: Every Addition Hurts

Strict ablation on MemoryAgentBench (20,000 bootstrap samples + Holm correction):

Remove reranking : −3.92 points — reranking is the critical component.

Return only 1 evidence : −11.73 points; k=5–60 stable, k=120 drops 14.57 points.

Add atomic fact extraction : −19.62 points (test‑time learning ability collapsed from 47.68 to 15.37).

Add summarization : −5.81 points.

Add graph‑extended retrieval : −5.78 points.

All “memory management” additions degrade performance. Compression and rewriting at ingestion discard information that retrieval cannot recover.

Why Naive RAG Works: Three Mechanistic Findings

The paper’s most valuable contribution is its mechanistic analysis:

Finding 1: The bottleneck is evidence coverage, not evidence utilization. RAW-RAG achieves the highest coverage (45.29% / 76.35%), while utilization rates across systems are similar — differences stem from whether evidence can be retrieved at all.

Finding 2: Raw history preserves reasoning paths. Retrieving the “answer value” alone is insufficient; temporal and dependency paths are needed. RAW-RAG’s conditional path coverage is 62.34% vs. MemU’s 9.62%.

Finding 3: Information lost at ingestion cannot be recovered at retrieval. Under an Oracle condition (full memory + gold‑standard location), RAW-RAG and MemMachine (which retain raw content) recover to 57.5%, close to the raw‑evidence upper bound of 61.6%; other systems lag by up to −30.5 points.

Paper: Is Complex Agentic Memory Always Needed? Naive RAG Is Stronger Than We Thought (arXiv:2610.30755)<br/> Code: https://github.com/Litmeb/RAW-RAG