Why Do Large Language Models Hallucinate and How to Reduce It?

Why Large Models Hallucinate

When using large models, users often encounter nonsensical or fact‑incorrect answers, a phenomenon called “hallucination” (translated from the English term “hallucination”). This arises from both data‑level problems—such as erroneous or biased pre‑training data and conflicting alignment data—and algorithmic issues during training and inference.

For example, if pre‑training data says Apple’s CEO is Tim Cook while alignment data says it is Bill Gates, the model becomes confused and may produce a hallucinated answer when asked about Apple’s CEO.

Additionally, models suffer from knowledge forgetting during pre‑training and stochastic decoding during inference, which further increase the chance of incorrect outputs.

How to Effectively Reduce Hallucinations

Solutions are grouped into data‑level, model‑level, and post‑processing methods.

Data‑level: Clean and filter training data to remove noise, bias, duplication, and factual errors; use manual annotation or heuristic rules to improve data quality.

Model‑level: Incorporate feedback (e.g., reinforcement learning with hallucination‑aware rewards), inject external knowledge via retrieval or knowledge bases, and apply constraints during decoding to force inclusion of required entities.

Post‑processing: Detect and correct hallucinations after generation by training a verifier on synthetic error data, especially for entities, numbers, and dates.

“Truth‑seeking” Algorithm

Tencent’s Hunyuan model introduces a “Truth‑seeking” algorithm during pre‑training that intervenes in the model’s hidden states when a potential hallucination is detected. A classifier inside the algorithm identifies patterns associated with hallucinations and adjusts the hidden variables during inference, achieving a 30%‑50% reduction in hallucination rate on the TruthfulQA benchmark without causing catastrophic forgetting.

Although hallucinations cannot be completely eliminated due to the inherent nature of transformer architectures, ongoing research—including reinforcement‑learning‑based trap detection, chain‑of‑thought prompting, and the “Truth‑seeking” technique—aims to further lower their occurrence.