SAG: The New RAG SOTA That Delivers Sub‑Second Retrieval on 500 Million Records

Traditional Retrieval‑Augmented Generation (RAG) systems excel at finding a few similar passages but often ignore the relationships between those passages, leading to hallucinations and poor multi‑hop reasoning. The article explains this limitation with a medical‑text study where hallucination rates jumped from 5.0% to 43.6% when only vector similarity was used.

SAG (SQL‑Retrieval Augmented Generation) addresses the problem by restructuring raw text into event + entity records stored in a relational database and a vector index. Each event captures a complete factual chain (who, when, what, outcome) while entities link events together, forming many‑to‑many hyperedges. During a query, SAG first runs a lightweight SQL join to retrieve relevant events, then expands the result set with additional events that share entities, and finally applies a vector similarity filter before the LLM generates the answer.

The article walks through a concrete example: answering “Why would investors fund *Letter to Grandma*?” SAG first identifies entities such as "Shenzhen company" and "regional culture", retrieves the corresponding event cards via SQL, expands to related events (cultural value, low‑cost production), and lets the LLM synthesize a concise justification. This two‑step process—structural path followed by semantic path—demonstrates why SAG’s recall improves early in the candidate list.

Benchmark experiments on HotpotQA, 2WikiMultiHopQA, and MuSiQue show that SAG achieves an average Recall@2 / Recall@5 of 79.3% / 88.2%, surpassing HippoRAG 2 (68.2% / 83.3%) and beating GraphRAG on multi‑hop reasoning. Ablation studies reveal that the hypergraph structure contributes most of the gain, while stronger embeddings (NV‑Embed‑v2) only marginally improve performance.

Scalability is a key focus: SAG has been deployed on a production corpus of over 500 million records with sub‑second online latency. The system separates heavy offline processing (event extraction, hyperedge construction) from lightweight online retrieval (SQL + vector lookup), allowing incremental updates without rebuilding the entire graph.

Beyond pure retrieval, the article argues that SAG provides a natural memory layer for autonomous agents. By treating each event as a timestamped fact and linking entities as navigable road signs, agents can trace the provenance of answers, maintain long‑term state, and perform version‑aware reasoning—capabilities missing from conventional RAG pipelines.

In conclusion, SAG represents a new data‑organization paradigm for next‑generation agents: a hybrid of relational joins, hypergraph reasoning, and dense retrieval that delivers SOTA multi‑hop performance, industrial‑scale latency, and a foundation for persistent agent memory.