Boosting LLM Accuracy: From RAG to GraphRAG for Enterprise Metadata Retrieval

What is Retrieval‑Augmented Generation (RAG)

RAG combines a retrieval step with large‑language‑model (LLM) generation. First, relevant evidence is fetched from a large knowledge base; then the LLM constructs an answer using that evidence, which improves factual accuracy, timeliness, and reduces hallucinations.

Architecture

Challenges

Key challenges include balancing relevance and freshness of retrieval, vector‑based recall missing key information, difficulty handling complex knowledge structures, and gaps between semantic understanding and real user intent. Effective solutions require scenario‑specific tuning, improved retrieval algorithms, knowledge filtering, and occasional integration of LLM reasoning.

Evaluation

GraphRAG

GraphRAG extends RAG by incorporating a Knowledge Graph, enabling deeper entity relationships and global insights. Instead of pure vector retrieval of document fragments, GraphRAG builds a structured graph from massive data, applies graph algorithms (e.g., community detection, centrality), and retrieves sub‑graphs or patterns as context for LLM generation.

Architecture (offline & online)

Offline stage: Raw knowledge is chunked into text blocks; an LLM extracts entities and relations; embeddings are stored in a Vector DB, while entities and relations are stored in a Graph DB.

Online stage: A user query is embedded, then hybrid retrieval (vector + graph) fetches relevant text blocks and graph entities/relations. The retrieved information is merged into a prompt for the LLM to generate answers for recommendation, retrieval, or dialogue scenarios.

Multi‑index combination: graph index, vector index, full‑text index

Hybrid retrieval: vector, full‑text, scalar

Multi‑hop reasoning: graph‑based knowledge enables multi‑step inference for complex queries

Comparison with traditional RAG

Graph‑based RAG variants

Three typical paradigms are GraphRAG (Microsoft open‑source), LightRAG, and PathRAG. They address knowledge mining, lightweight deployment, and path reasoning respectively, allowing flexible combination to meet diverse business scenarios.

Metadata Retrieval Use‑Case

Business background

Users often need to locate tables, fields, and business definitions. Traditional keyword search requires repeated back‑and‑forth with technical teams. A RAG‑enabled conversational QA can lower the barrier, reduce communication overhead, and improve data‑retrieval efficiency.

Key challenges

Solution 1.0 – Naive RAG

Effect: Overall performance below expectations; answer accuracy 55%, Top‑K hit rate ~60%.

Problem attribution:

Knowledge base contained only schema and comments, lacking business context, data lineage, etc.

Retrieval relied solely on vector search, failing on synonyms, multi‑entity queries, and table relationships.

No boundary awareness; out‑of‑scope queries produced misleading answers.

Index process (knowledge ingestion and chunking)

Retrieval process (vector search → LLM generation)

Chunking strategies

Option 1 (not used): Whole‑table chunk – preserves completeness but leads to (1) high similarity matches to irrelevant tables and (2) token length exceeding LLM limits.

Option 2 (used): Field‑level chunk – avoids the above issues; may increase chunk count but improves relevance and stays within token limits.

Example interaction

1. User question: "Which table contains the field xxx?"

2. Vector DB retrieval results:
{"tableName": "table1", "column": "xxx", "type": "string", "comment": "..."}
... (multiple JSON rows)

3. LLM generated answer:
"The field xxx can be found in table1."

Bad‑case analysis of Solution 1.0

Knowledge recall insufficient – relevant information fails to reach LLM context.

Knowledge‑base quality limited – missing business semantics and coverage.

Boundary definition unclear – leads to processing bias and hallucinations.

Typical failure patterns include semantic mismatch (e.g., synonym gaps), multi‑entity recall gaps, and interference from unrelated information.

Solution 2.0 – GraphRAG

Metadata naturally fits a graph representation. LightRAG was selected for its balance of complexity, flexibility, and embeddability.

Knowledge‑base construction

A progressive expansion strategy starts from core domains (e.g., order domain) and gradually extends to the entire warehouse.

Graph storage design

Three entity types are stored:

Table/field nodes with type, description, and relationships (including lineage).

Business‑term nodes for domain terminology.

Synonym‑layer nodes linked by edges to capture equivalent expressions.

Vector storage design

Indexing workflow

Multiple sources (tables, metadata platform, data lineage) are transformed into entities and relationships, combined with LLM‑extracted terminology, and stored in Graph Storage. Document chunks are stored in Vector Storage. Entity IDs serve as primary keys for incremental updates.

Entity weight calculation

Table score: manual_boost_1 * (w1 × score_downstream + w2 × score_popularity + w3 × score_star)

Field score: manual_boost_2 * (w4 × base_score + w5 × table_factor)

Term score: 1

Retrieval workflow

User query is parsed by an LLM into high‑level and low‑level keywords. Low‑level keywords are expanded via a synonym dictionary, then hybrid retrieval (vector + BM25) and re‑ranking produce top‑K entities, which are linked to the graph for contextual enrichment (Local Query Context). High‑level keywords undergo embedding retrieval for top‑K relations, forming a Global Query Context. Both contexts are merged and fed to the LLM.

Effectiveness

Answer quality breakthrough: Overall accuracy rose from 56% to 78%; knowledge recall 91%; Top‑K hit rate 90%; Mean Reciprocal Rank (MRR) 0.73.

Business impact: AI‑driven retrieval achieved ~30% penetration over traditional keyword search, saving >20% time in data‑warehouse Q&A and reducing repetitive communication.

Future work

Retrieval upgrades: stronger hybrid retrieval, improved ranking models, and richer re‑ranking features (e.g., table importance, field usage frequency).

Knowledge‑base enrichment: expand domain terminology, automate metadata augmentation, and increase synonym coverage.

Agentic RAG: introduce planning agents, multi‑hop reasoning loops, and proactive clarification capabilities to handle ambiguous or incomplete queries.

In summary, moving from RAG to GraphRAG demonstrates that organizing metadata as a graph dramatically improves recall and accuracy; the remaining bottlenecks lie in retrieval and knowledge organization rather than the LLM itself.