Jev Decision Model Beats LLM Reranking in Graph RAG for Multi-Hop QA

Multi-hop question answering remains challenging because answers are scattered across multiple documents with implicit connecting clues. Graph RAG uses knowledge graphs to link these relations, but traversing the graph yields many candidates that slow down downstream LLM filtering.

Zilliz's open-source Vector Graph RAG combines the Jev decision model with graph retrieval, letting a lightweight decision model handle evidence selection. The system stores entities as nodes, relations as edges, and keeps the original text for each relation — all in Milvus, avoiding a separate graph database.

Five-Step Retrieval Walkthrough

Example question: "What side effects should be watched for first-line type 2 diabetes drugs?" The knowledge base contains treatment guidelines, drug labels, and research papers. The drug name is not in the question, so the system must first find the drug, then its adverse reactions.

Find starting points. Extract entities from the question, vector-search for relevant entities and relations. Semantic matches like "side effects" ↔ "adverse reactions" are captured.

Expand subgraph. Traverse from the matched nodes/edges to discover connected entities (e.g., "metformin") and their linked documents, bringing in names not mentioned in the question.

Jev stage 1: filter relations. Hundreds of relations (R&D, treatment, symptoms) may be retrieved. Jev scores each relation for relevance to the question, keeping only useful connections.

Retrieve and merge passages. Fetch source text for the retained relations, add passages from direct vector search, and deduplicate. This captures both graph-structured and unstructured evidence.

Jev stage 2: rerank passages. The same drug relation may appear in a guideline and a lengthy research history. Jev reads candidate passages and ranks those actually discussing side effects higher before sending to the answer model.

Both stages score candidates; the program filters and sorts by score, avoiding generation of long reranking outputs. Entity/relation extraction at ingestion and final answer generation still use generative models.

Benchmark Results

Evaluated on MuSiQue and 2WikiMultihopQA (1,000 questions each), testing the ability to recover evidence scattered across documents.

Two-stage Jev 1.14 achieves 86.61% average Recall@5 — highest recall among compared methods.

Jev 1.13 reaches 86.07% Recall@5 with lower model call latency.

On the effect-vs-latency plot, Jev-based configurations form a "kill line" that is both faster and more accurate, beating all LLM-reranking baselines.

Quick Start

1. Install Python package

uv init --python 3.12 jev-demo
cd jev-demo
uv add vector-graph-rag

Defaults to Milvus Lite (local file). For larger data, connect to Milvus Server or Zilliz Cloud.

2. Configure keys

VGRAG_OPENAI_API_KEY=your_openai_api_key
VGRAG_JEV_API_KEY=your_typesafe_api_key

OpenAI key used for embeddings, entity/relation extraction, and final answer; Typesafe key for Jev relation filtering and passage reranking.

3. Import texts and query

from vector_graph_rag import VectorGraphRAG

rag = VectorGraphRAG(
    milvus_uri="./demo.db",
    collection_prefix="jev_demo",
    reranker_model="jev-1.13.0"  # or jev-1.14.0
)

rag.rebuild_texts([
    "Galaxy A1 supports offline use. Galaxy B2 requires internet.",
    "Galaxy A1 warranty is three years. Galaxy B2 warranty is one year."
])

result = rag.query("Which products support offline use and have warranty over two years?")
print(result.answer)
print(result.passages)

Run with uv run python demo.py. rebuild_texts() does full ingestion; subsequent runs can omit it — data persists in demo.db.

4. Import PDF, Word, or Markdown

uv add "vector-graph-rag[loaders]"

from vector_graph_rag import VectorGraphRAG
from vector_graph_rag.loaders import DocumentImporter

importer = DocumentImporter(chunk_size=1000, chunk_overlap=200)
loaded = importer.import_sources([
    "./docs/product-manual.pdf",
    "./docs/warranty.docx",
    "./docs/faq.md"
])
if loaded.errors:
    raise RuntimeError("
".join(loaded.errors))
if not loaded.documents:
    raise ValueError("No text was extracted from the documents.")

rag = VectorGraphRAG(
    milvus_uri="./documents.db",
    collection_prefix="my_docs",
    reranker_model="jev-1.13.0"
)
rag.rebuild_documents(loaded.documents, extract_triplets=True)

result = rag.query("Which products support offline use and have warranty over two years?")
print(result.answer)
print(result.passages)

The loader handles file reading, text extraction, and chunking; rebuild_documents() performs entity/relation extraction and vector indexing; query() runs retrieval, two-stage Jev reranking, and answer generation.

Optional: REST API and Visualization Frontend

uv add "vector-graph-rag[api]"
uv run uvicorn vector_graph_rag.api.app:app --host 127.0.0.1 --port 8000

Add VGRAG_RERANKER_MODEL=jev-1.13.0 to .env to enable two-stage Jev. API provides document import and query endpoints; visit http://127.0.0.1:8000/docs for interactive testing.

The project includes a visualization frontend (in the frontend directory) to inspect subgraph expansion, relation filtering, source passages, and answers. Start with npm ci and npm run dev, then open http://localhost:5173 (requires the API service running).

Making Multi-Hop Retrieval Practical

Vector Graph RAG connects fragmented knowledge inside a vector database, eliminating the overhead of maintaining a separate graph database. Jev accelerates filtering and reranking, giving multi-hop retrieval a better trade-off between effectiveness and speed. The project also demonstrates a promising use case for decision models: using lightweight judgments to boost retrieval efficiency wherever large candidate sets must be pruned.