How GraphRAG Turns Knowledge Graphs into Smarter Retrieval for LLMs

1. Introduction: Why GraphRAG?

Large language models (LLMs) often hallucinate because they rely on static training data and cannot fetch up‑to‑date information. Retrieval‑Augmented Generation (RAG) mitigates this by first retrieving relevant documents and then prompting the model with the retrieved context. However, conventional RAG struggles with multi‑hop reasoning and entity linking.

GraphRAG addresses these limitations by converting the retrieved text into a knowledge graph (nodes = entities, edges = relationships). The graph enriches the context with related entities, enabling more accurate and explainable answers.

2. GraphRAG Detailed Process

Indexing : Documents are split into text units (chunks) with configurable size and overlap. Each chunk is embedded for similarity search.

Entity & Relationship Extraction : An LLM extracts entities (people, organizations, concepts, etc.) and the relations between them from each chunk.

Graph Construction : Extracted entities become nodes; relations become edges, forming a knowledge graph that preserves the original semantics.

Community Detection & Report Generation : The graph is clustered (Leiden algorithm) into communities. For each community a concise report is generated, summarizing its core nodes and connections.

Retrieval : Two query modes are available

Local retrieval : Embed the user question, find the most similar chunks, expand the candidate set by traversing the graph, rank by relevance, and feed the concatenated context to the LLM.

Global retrieval : Retrieve community reports first, let the LLM generate a provisional answer for each community (Map phase), rank and filter the provisional answers, concatenate them, and produce the final answer (Reduce phase).

3. Practical Walkthrough

3.1 Environment Setup

Install the required packages (Python 3.10‑3.12): pip install graphrag Install pandas for parquet handling: pip install pandas Create a project directory (e.g., rag_demo) and place source .txt files under rag_demo/input/. Example dataset (sorting‑algorithm description) is provided in the article.

3.2 Project Initialization

Run the GraphRAG initializer: graphrag init --root ./rag_demo The command creates the following structure (relevant parts only):

rag_demo/</code>
<code>├─ input/</code>
<code>│   └─ file.txt</code>
<code>├─ prompts/</code>
<code>│   ├─ basic_search_system_prompt.txt</code>
<code>│   ├─ community_report_graph.txt</code>
<code>│   └─ … (other prompt templates)</code>
<code>├─ settings.yaml</code>
<code>└─ .env

settings.yaml

contains LLM and embedding model configuration (e.g., gpt‑4.1 for chat, text‑embedding‑3‑large for embeddings) and chunk parameters ( size: 50, overlap: 10). The .env file stores GRAPHRAG_API_KEY and GRAPHRAG_BASE_URL.

3.3 Index Construction

Build the index with: graphrag index --root ./rag_demo The process creates a cache/ folder for intermediate LLM calls, a logs/ file, and an output/ directory containing parquet files ( text_units.parquet, entities.parquet, relationships.parquet, communities.parquet, community_reports.parquet, etc.). These files store the chunk embeddings, extracted graph components, and community metadata.

3.4 Retrieval Examples

Local retrieval (question: “How many sorting algorithms are mentioned in the text and what are they?”):

graphrag query --root ./rag_demo --method local --query "How many sorting algorithms are mentioned and what are they?"

Global retrieval (same question):

graphrag query --root ./rag_demo --method global --query "How many sorting algorithms are mentioned and what are they?"

Both commands return a concise answer that lists all eight algorithms (Bubble Sort, Quick Sort, Shell Sort, Merge Sort, Heap Sort, Radix Sort, Counting Sort, Pigeonhole Sort) and the total count.

3.5 GraphRAG vs. Traditional RAG

A side‑by‑side experiment using Spring AI RAG (chunk size = 50, similarity threshold = 0.8) shows that the traditional pipeline misses five algorithm mentions because their similarity scores fall below the threshold. GraphRAG, by traversing the graph and pulling related entities, retrieves the missing context and produces a correct answer.

3.6 Visualization with Neo4j

To visualise the graph, import the parquet files into Neo4j. Install Neo4j (e.g., brew install neo4j) and the APOC plugin. Then run the following Python script (requires neo4j driver):

pip install neo4j

import pandas as pd
from neo4j import GraphDatabase
import time

def batched_import(statement, df, batch_size=1000):
    total = len(df)
    start_s = time.time()
    for start in range(0, total, batch_size):
        batch = df.iloc[start:min(start + batch_size, total)]
        result = driver.execute_query(
            "UNWIND $rows AS value " + statement,
            rows=batch.to_dict('records'),
            database_=NEO4J_DATABASE)
        print(result.summary.counters)
    print(f"{total} rows in {time.time() - start_s} s.")
    return total

NEO4J_URI = "neo4j://localhost"
NEO4J_USERNAME = "neo4j"
NEO4J_PASSWORD = "neo4j"
NEO4J_DATABASE = "neo4j"

driver = GraphDatabase.driver(NEO4J_URI, auth=(NEO4J_USERNAME, NEO4J_PASSWORD))
GRAPHRAG_FOLDER = "./rag_demo/output"

text_units = pd.read_parquet(GRAPHRAG_FOLDER + "/text_units.parquet")
entities = pd.read_parquet(GRAPHRAG_FOLDER + "/entities.parquet")
relationships = pd.read_parquet(GRAPHRAG_FOLDER + "/relationships.parquet")
communities = pd.read_parquet(GRAPHRAG_FOLDER + "/communities.parquet")
community_reports = pd.read_parquet(GRAPHRAG_FOLDER + "/community_reports.parquet")

# Import text units (chunks)
statement = """
MERGE (c:__Chunk__ {id:value.id})
SET c += value {.text, .n_tokens}
WITH c, value
UNWIND value.document_ids AS document
MATCH (d:__Document__ {id:document})
MERGE (c)-[:PART_OF]->(d)
"""
batched_import(statement, text_units)

# Import entities
entity_statement = """
MERGE (e:__Entity__ {id:value.id})
SET e += value {.human_readable_id, .description, name:replace(value.title,'"','')}
WITH e, value
CALL apoc.create.addLabels(e, case when coalesce(value.type,"") = "" then [] else [apoc.text.upperCamelCase(replace(value.type,'"',''))] end) YIELD node
UNWIND value.text_unit_ids AS text_unit
MATCH (c:__Chunk__ {id:text_unit})
MERGE (c)-[:HAS_ENTITY]->(e)
"""
batched_import(entity_statement, entities)

# Import relationships
relationships_statement = """
MATCH (source:__Entity__ {name:replace(value.source,'"','')})
MATCH (target:__Entity__ {name:replace(value.target,'"','')})
MERGE (source)-[rel:RELATED {id: value.id}]->(target)
SET rel += value {.weight, .human_readable_id, .description, .text_unit_ids}
RETURN count(*) as createdRels
"""
batched_import(relationships_statement, relationships)

# Import communities
communities_statement = """
MERGE (c:__Community__ {id:value.id})
SET c += value {.level, .title, .community}
WITH *
UNWIND value.relationship_ids as rel_id
MATCH (start:__Entity__)-[:RELATED {id:rel_id}]->(end:__Entity__)
MERGE (start)-[:IN_COMMUNITY]->(c)
MERGE (end)-[:IN_COMMUNITY]->(c)
RETURN count(distinct c) as createdCommunities
"""
batched_import(communities_statement, communities)

# Import community reports
community_reports_statement = """
MERGE (c:__Community__ {community:value.community})
SET c += value {.level, .title, .rank, .rank_explanation, .full_content, .summary}
WITH c, value
UNWIND range(0, size(value.findings)-1) AS finding_idx
WITH c, value, finding_idx, value.findings[finding_idx] as finding
MERGE (c)-[:HAS_FINDING]->(f:Finding {id:finding_idx})
SET f += finding
"""
batched_import(community_reports_statement, community_reports)

After importing, Neo4j can be queried to explore the full graph, e.g., list all nodes and relationships, find entities linked to "Bubble Sort", or retrieve all entities belonging to community 2.

4. Conclusion

GraphRAG’s graph‑centric approach provides richer context, better multi‑hop reasoning, and higher answer accuracy compared with plain vector‑based RAG, especially for complex or cross‑document queries. The trade‑off is additional computational cost for entity extraction and graph construction, and the need to rebuild the graph when the source corpus changes. As LLM capabilities and automated graph‑building techniques improve, these costs are expected to diminish, making GraphRAG a promising foundation for next‑generation knowledge‑aware AI systems.