Advanced Graph RAG with Neo4j: When Multi‑Hop Reasoning Beats Vector Search

1. Vector Retrieval Blind Spot: It Doesn't Understand "Between"

Vector search embeds a query and returns the nearest document fragments, which captures semantic similarity but not logical connections. In a supply‑chain scenario, the answer to "Do Supplier A and recall batch C relate?" requires a path across four entities that are stored in separate documents, a situation vector search cannot resolve because it lacks a notion of "path".

Typical multi‑hop scenarios (e.g., supply‑chain risk, medical knowledge, legal precedent, organizational queries) require two or more hops to answer.

2. Multi‑Hop Reasoning as Graph Traversal

In a knowledge graph, multi‑hop reasoning is essentially a path‑traversal problem. Cypher expresses this concisely:

-- 1‑hop: direct relation
MATCH (s:Supplier)-[:PROVIDES]->(p:Part)
WHERE s.name = "Supplier A"
RETURN p.name

-- 2‑hop: supplier → part → product
MATCH (s:Supplier)-[:PROVIDES]->(p:Part)-[:USED_IN]->(prod:Product)
WHERE s.name = "Supplier A"
RETURN s.name, p.name, prod.name

-- 3‑hop: supplier → part → product → recall batch
MATCH (s:Supplier)-[:PROVIDES]->(p:Part)-[:USED_IN]->(prod:Product)-[:IN_RECALL]->(r:RecallBatch)
WHERE s.name = "Supplier A"
RETURN s.name, p.name, prod.name, r.batchId, r.reason

-- Variable hops (1..4)
MATCH path = (s:Supplier)-[*1..4]->(r:RecallBatch)
WHERE s.name = "Supplier A"
RETURN path

The Cypher pattern -[*1..4]-> lets the query declare a variable‑length path, something vector databases cannot express.

3. Building a Multi‑Hop Graph: Modeling Matters

Retrieval quality depends heavily on graph modeling. Relationships must be fine‑grained and intermediate nodes must be retained; otherwise the reasoning path is broken.

-- ❌ Bad: collapse intermediate steps
CREATE (s:Supplier {name: "Supplier A"})
CREATE (r:RecallBatch {id: "Q3-2024"})
CREATE (s)-[:RELATED_TO]->(r)

-- ✅ Good: keep each hop with attributes
CREATE (s1:Supplier {name: "Supplier A", country: "China"})
CREATE (p1:Part {id: "PART-001", name: "Brake Pad", criticalLevel: "HIGH"})
CREATE (prod1:Product {id: "MODEL-X", name: "X Series Car"})
CREATE (r1:RecallBatch {id: "Q3-RECALL-001", reason: "Brake failure", count: 15000})

CREATE (s1)-[:PROVIDES {since: "2022-01", quality: "B+"}]->(p1)
CREATE (p1)-[:USED_IN {quantity: 4, position: "front"}]->(prod1)
CREATE (prod1)-[:IN_RECALL {affectedCount: 8000}]->(r1)

Use MERGE instead of CREATE to make the ingestion idempotent and avoid duplicate nodes.

4. From Natural Language to Cypher (NL2Cypher)

Hand‑writing Cypher is feasible for experts, but end‑users need the LLM to generate it. The default prompt of GraphCypherQAChain yields uneven quality for complex multi‑hop queries, so a few‑shot prompt is added:

const CYPHER_GENERATION_TEMPLATE = `
You are a Neo4j Cypher expert. Given the graph schema and a question, generate an exact Cypher query.

Schema:
{schema}

Rules:
1. Return only the Cypher statement.
2. Prefer MATCH path = ... for multi‑hop queries.
3. Always add LIMIT to avoid full‑graph scans.

Few‑shot examples:
Q: Which suppliers provide parts used in recalled products?
A: MATCH (s:Supplier)-[:PROVIDES]->(p:Part)-[:USED_IN]->(prod:Product)-[:IN_RECALL]->(r:RecallBatch) RETURN DISTINCT s.name AS supplier, p.name AS part, prod.name AS product, r.id AS recall

Q: What is the full path from a supplier to a recall batch?
A: MATCH path = (s:Supplier)-[*1..4]->(r:RecallBatch) RETURN path LIMIT 20

Q: {question}
A:`;

Running the chain with verbose and returnIntermediateSteps lets developers see the generated Cypher and debug failures.

5. Hybrid Retrieval: Graph + Vector

Graph traversal excels at structured relationship reasoning, while vector search handles fuzzy semantic matching. Combining both yields a production‑grade RAG pipeline.

async function hybridRetrieve(question: string): Promise<string> {
  const [vectorDocs, graphContext] = await Promise.all([
    vectorRetriever.invoke(question),
    graphRetrieve(question),
  ]);
  const vectorContext = vectorDocs.map(d => d.pageContent).join("
");
  return `【Semantic Retrieval】
${vectorContext}

【Graph Path】
${graphContext}`;
}

The final prompt feeds the merged context to the LLM, which prefers the deterministic graph path and falls back to semantic snippets for additional detail.

6. When to Choose Graph RAG vs Vector RAG

Key comparison dimensions (cost, latency, multi‑hop capability, explainability, update cost, data type, global aggregation, index cost) show that Graph RAG is justified only when:

Data is reused frequently (high query volume).

Answers must be auditable with explicit paths (compliance, regulatory).

Questions involve pattern or aggregation queries across the graph.

Otherwise, a vector‑only RAG with a reranker is more cost‑effective.

7. Common Pitfalls

Schema too coarse: Using a generic RELATED_TO relation loses type information; define precise types like PROVIDES, USED_IN, IN_RECALL.

LLM‑generated Cypher errors: Run in verbose mode, collect failing queries as few‑shot negatives, and maintain a regression test set.

Unbounded variable‑length paths: Omitting LIMIT can cause full‑graph scans and timeouts; always cap the hop range.

Insufficient node descriptions: Relying only on name harms vector recall; enrich description with contextual information.

Incorrect relationship direction: Cypher respects direction; inconsistent direction during modeling leads to empty results.

Conclusion

Vector retrieval's blind spot is the "path" – it cannot perform multi‑hop logical reasoning.

Cypher's [*1..N] syntax is the core weapon for multi‑hop Graph RAG.

Graph modeling quality (fine‑grained relations, retained intermediate nodes, rich descriptions) determines retrieval effectiveness.

Hybrid retrieval (graph + vector) is the production‑grade approach.

Adopt Graph RAG only when high query frequency, auditability, and aggregation needs outweigh its ~1000× higher indexing cost.