Taming LLM Hallucinations: Strategies and Solutions from 360

1. What Is a Large‑Model Hallucination?

Large‑model hallucination refers to the phenomenon where a model generates fluent, low‑perplexity text that is factually incorrect or unverifiable, often described as "serious nonsense". It can be classified into two major categories: factual hallucinations (including factual inconsistency and fabricated facts) and faithful hallucinations (including failure to follow instructions or context).

Classification Flow

In practice, one first checks whether the model follows user instructions and context; violations indicate faithful hallucinations. Then, assess the correctness of the response. Errors can stem from knowledge gaps, outdated information, or reasoning mistakes.

2. Causes of Hallucinations

Data: missing, outdated, or misaligned knowledge.

Algorithm & Training: decoder‑only architecture, attention limitations, exposure bias between training (teacher forcing) and inference (autoregressive generation), and alignment fine‑tuning that may encourage over‑confidence.

Inference: stochastic decoding (temperature, top‑k, top‑p), long‑context truncation, and softmax bottleneck.

3. Detecting Hallucinations

Knowledge‑Determinism Classification

First determine if a question has a deterministic answer. For non‑deterministic queries, provide multiple plausible answers.

Self‑Consistency Checks

Generate multiple answers to the same question and examine consistency; divergent answers suggest possible hallucination.

External Tool Verification

Use search engines, code interpreters, or knowledge bases to retrieve evidence and compare with model output.

4. Mitigation Strategies

RAG (Retrieval‑Augmented Generation)

Incorporate external knowledge via a pipeline: query analysis → retrieval (keyword, semantic, structured) → ranking → context selection → generation. This reduces reliance on internal memorization.

Decoding Strategies

Dynamic adjustment of decoding parameters based on task.

Contrastive decoding: compare outputs of a large model with a smaller model to prune unreliable tokens.

Recitation‑augmented generation: prompt the model to recall relevant knowledge before answering.

Self‑Critique & Editing

Apply critic models (e.g., RARR, FAVA, Gen‑Critic‑Edit) to rewrite queries, verify answers against retrieved evidence, and iteratively refine outputs.

Model‑Enhancement Techniques

Pre‑training with up‑to‑date data.

Fine‑tuning and alignment (SFT, DPO) with carefully curated positive/negative examples.

5. 360’s Trusted Large‑Model Solutions

360 applied the above methods in content‑safety detection, achieving top rankings in the AI Safety Benchmark, and integrated them into products such as 360AI Search and 360AI Browser, which demonstrate reduced hallucination in real‑world scenarios.

6. Future Exploration

Continuous benchmarking in production environments, building domain‑specific benchmarks, and iterating on retrieval, alignment, and evaluation pipelines are essential for advancing trustworthy LLM deployment.

7. Q&A

Q1: How is the binary classifier in the RAG workflow trained? A1: By constructing a hallucination taxonomy, labeling data from online logs using automated tools and human annotation, then training a classification model.

Q2: What methods are used for re‑ranking mixed retrieval results? A2: Strategies include keyword search, semantic search (e.g., bge‑rerank), database and graph search, followed by rule‑based or learned re‑ranking models tailored to the business scenario.