GraphRAG Deep Dive: Boost Multi‑Hop Reasoning Accuracy from 50% to 85% with Knowledge Graphs

01 The Fundamental Flaw of Traditional RAG: Lost Relational Information

Traditional vector RAG splits documents into isolated chunks, discarding the connections between entities. An example with three documents about a CTO, a strategic partnership, and a CEO shows that a vector‑only query returns only the most similar chunk, missing the full relationship chain.

The problem is that vector RAG treats each chunk as independent, losing the relational graph needed for multi‑hop inference.

GraphRAG solves this by constructing a graph:

李四 → [CEO] → ABC公司 → [合作于] → XYZ公司 → [CTO] → 张三 → [主导] → 云迁移项目

, enabling two‑hop reasoning.

02 Core Components of GraphRAG: Graph Construction + Graph Retrieval

The workflow consists of an indexing stage (graph building) and a query stage (graph traversal).

【索引阶段】                         【查询阶段】
原始文档                             用户问题
   ↓                                    ↓
实体关系抽取（LLM 驱动）              实体识别（提取关键实体）
   ↓                                    ↓
知识图谱构建                         图遍历（沿边扩展 1-3 跳）
（Entity + Relation + Attribute）        ↓
   ↓ 同时                            相关子图召回（连通节点集合）
向量索引（Entity 的 Embedding）        ↓
   ↓                                拼装上下文 → LLM 生成答案
存入 Neo4j

During indexing, LLM extracts entity‑relation‑attribute triples and stores them in Neo4j. During querying, the system identifies entry entities from the question, performs 1‑3‑hop graph traversal, and assembles the retrieved sub‑graph as context for the LLM.

03 Implementing GraphRAG with LangChain + Neo4j

Install dependencies:

npm install @langchain/community @langchain/openai langchain neo4j-driver

Key code snippets include initializing Neo4j, configuring LLMGraphTransformer with allowed node and relationship types, converting documents to graph documents, creating a vector store for entity embeddings, and defining a custom GraphRetriever that first performs semantic search for entry entities and then runs a Cypher multi‑hop query.

class GraphRetriever extends BaseRetriever {
  async _getRelevantDocuments(query) {
    const entities = await vectorStore.similaritySearch(query, 3);
    const entityNames = entities.map(e => e.metadata.id);
    const results = await graph.query(`
      MATCH (e:__Entity__)-[r:MANAGES|WORKS_FOR|COLLABORATED_WITH|DEVELOPED_BY*1..2]-(n)
      WHERE e.id IN $entities
      RETURN DISTINCT n.id as entity, n.text as content, labels(n)[0] as type
      LIMIT 20
    `, { entities: entityNames });
    return results.map(r => new Document({
      pageContent: r.content || r.entity,
      metadata: { entity: r.entity, type: r.type, source: "graph" }
    }));
  }
}

The resulting answer correctly links the CEO, the partnership, and the CTO’s cloud‑migration project—something traditional vector RAG cannot produce.

04 Retrieval Modes: Local vs. Global

Microsoft’s GraphRAG paper defines two modes:

Local Search : 1‑3‑hop traversal from specific entities, ideal for precise relational reasoning (accuracy ★★★★★ 80‑85%, latency ~1 s, low cost).

Global Search : Community detection (Louvain) on the whole graph with summary generation, suited for macro‑analysis (accuracy ★★★★☆ 70‑75%, latency ~5 s, higher cost).

Traditional vector RAG serves as a baseline (accuracy ★★★☆☆ 45‑50%, extremely low latency and cost).

In practice, many systems blend Local Search with vector RAG to cover a broader range of queries.

05 Hybrid Retrieval: Combining GraphRAG and Vector RAG

GraphRAG complements rather than replaces vector RAG. A parallel‑pipeline ensemble can be built with EnsembleRetriever, weighting the graph retriever higher when relational reasoning is primary.

const hybridRetriever = new EnsembleRetriever({
  retrievers: [vectorStore.asRetriever({k:5}), new GraphRetriever()],
  weights: [0.4, 0.6]
});

Weight adjustments such as graph:vector = 4:6 (fact‑focused) or graph:vector = 7:3 (relation‑focused) let the LLM decide dynamically.

06 Three Crucial Factors for Graph Quality

Entity extraction granularity : 3‑8 entities per 200‑400 words is a good range; use allowedNodes and strictMode: true to enforce it.

Semantic consistency of relationship labels : Standardize labels (e.g., RELATED_TO) to avoid fragmented edges.

Rich node attributes : Store the original text fragment ( includeSource: true) and regularly de‑duplicate entities with similarity > 0.95.

07 Common Pitfalls and Solutions

Graph size explosion : Limit returned nodes with LIMIT and create indexes on e.id and e.embedding in Neo4j.

Duplicate entity names : After graph construction, cluster entities by embedding similarity > 0.95 and merge their edges.

Context overload : Restrict Cypher to a whitelist of relationship types and apply LIMIT 15 to keep context concise.

High indexing cost : Use gpt-4o-mini instead of gpt-4o, batch documents (10 per call), and perform incremental updates to cut costs to about one‑eighth.

Summary

Traditional RAG loses relational information because it treats chunks as isolated.

GraphRAG replaces similarity ranking with structured triples and graph traversal, raising multi‑hop accuracy from ~50% to ~85%.

Choose Local Search for precise reasoning and Global Search for thematic analysis; they are not mutually exclusive.

Graph quality hinges on entity granularity, consistent relationship labels, and rich node attributes with de‑duplication.

The most robust production setup mixes GraphRAG and vector RAG in a parallel ensemble, adjusting weights per query type.