How Large Language Models and Knowledge Graphs Can Boost Each Other

Comparison of Large Language Models and Knowledge Graphs

Large language models (LLMs) are built on deep neural networks and excel at natural‑language understanding and generation. Their knowledge is stored implicitly in model parameters, which can lead to hallucinations and limited interpretability. Knowledge graphs (KGs) represent facts explicitly as structured triples, offering high interpretability and reliable reasoning, but they are expensive to construct and perform poorly on raw text tasks. The complementary strengths suggest a synergistic combination.

LLM‑driven Knowledge Extraction

Instruction tuning enables LLMs to follow task‑specific prompts and extract structured information such as entities, relations, and events from raw text. Representative systems include:

InstructUIE – a multi‑task instruction‑tuned model for unified information extraction (Wang & Zhou, 2023). http://arxiv.org/abs/2304.08085 KnowLM – a knowledge‑aware LLM that can be fine‑tuned with supervised feature tuning (SFT) for higher extraction accuracy. https://github.com/zjunlp/KnowLM These approaches allow a single LLM to perform entity, relation, and event extraction without task‑specific model training.

LLM‑driven Knowledge Completion

Research extracts latent parametric knowledge from LLMs and converts it into structured KG triples to enrich existing graphs. Because LLM outputs may be hallucinated, specialized fine‑tuning or constraint‑based training (e.g., alignment with KG schemas) is required to obtain reliable completions. This area remains open for further investigation.

Knowledge Graphs for LLM Evaluation

The KoLA benchmark assesses LLM world‑knowledge across four levels: memory, understanding, application, and innovation. Experiments reveal:

LLMs still exhibit substantial gaps on all four levels.

Model size correlates positively with raw memorization ability.

After instruction fine‑tuning, higher‑order abilities (application, innovation) improve with size, while lower‑order abilities (memory, understanding) may decline—a phenomenon termed “alignment tax.”

Knowledge Graphs Enhancing LLM Applications

Accuracy and Explainability – Feeding KG‑derived facts as external context reduces hallucinations and yields more accurate, traceable answers.

Safety and Consistency – Post‑generation verification against KG triples can detect contradictions (e.g., anachronistic statements) and filter unsafe content.

Complex Reasoning – Structured KG data supports multi‑hop reasoning. The KoPL programming language translates natural‑language queries into composable functions that operate over KG facts, enabling precise reasoning and transparent execution.

Interactive Fusion of Knowledge Graphs and LLMs

Future research envisions a bidirectional loop: KGs provide structured knowledge to guide LLM generation, while LLMs continuously extract new facts from text to expand and refine KGs. Iterative collaboration can deepen semantic understanding, broaden coverage, and strengthen reasoning capabilities.

References

Wang X, Zhou W. InstructUIE: Multi‑task Instruction Tuning for Unified Information Extraction. arXiv 2023. http://arxiv.org/abs/2304.08085 zjunlp/KnowLM. GitHub repository, 2024. https://github.com/zjunlp/KnowLM Xie X et al. From Discrimination to Generation: Knowledge Graph Completion with Generative Transformer. WWW 2022. http://arxiv.org/abs/2202.02113 Yu J et al. KoLA: Carefully Benchmarking World Knowledge of Large Language Models. arXiv 2023. http://arxiv.org/abs/2306.09296 Shi W et al. REPLUG: Retrieval‑Augmented Black‑Box Language Models. arXiv 2023. http://arxiv.org/abs/2301.12652 Cao S et al. KQA Pro: A Dataset with Explicit Compositional Programs for Complex Question Answering over Knowledge Base. arXiv 2022.

http://arxiv.org/abs/2007.03875