How Baidu Search Is Transforming Machine Question Answering with Large‑Scale AI Models

Machine question answering (QA) enables software to automatically respond to natural‑language queries, e.g., answering "What is the name of the show hosted by Wang Xiaoya?" directly in Baidu Search’s top result, bypassing traditional keyword‑based retrieval.

The field has progressed in parallel with machine learning: before 2013, systems relied on handcrafted features and lexical matching (BM25); 2014‑2015 introduced neural networks (CNN, RNN) for semantic similarity; 2016‑2017 added attention mechanisms; 2018‑2021 focused on large pre‑trained models for complex matching; and since 2022 generative models have become dominant.

Corresponding dataset milestones include MCTest (2013), SQuAD (2016), Baidu’s DuReader (2017), HotpotQA and related multi‑hop/commonsense benchmarks (2018), each driving richer QA capabilities.

Modern QA pipelines adopt a Retriever + Reader paradigm. Baidu Search provides a powerful Retriever that returns diverse candidates (webpages, videos, tables, knowledge graphs). The research focus therefore shifts to the Reader, which extracts or generates answers from the retrieved material.

Early Readers followed a complex pipeline: query analysis → candidate generation → hand‑crafted matching features → ranking → answer extraction. This multi‑stage process accumulated errors and hindered end‑to‑end training, prompting a move toward Machine Reading Comprehension (MRC) models that directly map Question + Document to Answer, exemplified by BiDAF’s LSTM‑based encoding and bidirectional attention.

Subsequent advances replaced elaborate architectures with transformer‑based pre‑trained models (BERT, ERNIE), simplifying MRC while improving performance through large‑scale language understanding.

Baidu’s current DocMRC model illustrates hierarchical modeling: the entire document is split into sentences, each prefixed with a special token; a shallow word‑level encoder captures local representations, a hierarchical layer learns deep context, and the CLS token aggregates sentence information for answer prediction. Two output heads support multi‑sentence summaries and entity‑level span extraction.

Three core challenges remain: (1) deep semantic understanding and reasoning, (2) low‑latency response under massive traffic, and (3) ensuring answer correctness amid noisy web sources. Solutions include deploying hundred‑billion‑parameter models for richer knowledge, long‑sequence modeling for context, and knowledge distillation.

Knowledge distillation is performed in three stages: (i) multi‑teacher training to raise the learning ceiling, (ii) unsupervised distillation using teacher voting to filter noisy teachers, and (iii) supervised distillation with dynamic teacher weighting based on labeled data, yielding a student model that often surpasses individual large teachers.

To mitigate hallucinations and improve answer quality, Baidu employs Retrieval‑Augmented Generation (RAG): (1) retrieve multiple reference documents, (2) extract key information, (3) construct prompts that request numbered citations, and (4) generate answers with a large language model. This workflow produces concise, citation‑rich responses even for multi‑document queries.

Complex instruction handling is addressed by decomposing a high‑level request into three simple steps—select relevant results, organize/generate the answer, and attach numbered sources—allowing the model to learn each sub‑task separately and generalize to the full instruction.

Additional inference acceleration techniques include "Inference with Reference" (copying matching prefixes from retrieved texts to enable parallel decoding) and a two‑model scheme where a small model proposes candidate tokens that are verified by the large model, reducing latency without sacrificing accuracy.

Finally, the article poses an open question about the future shape of search engines, inviting readers to contribute ideas and submit resumes to Baidu’s recruitment channel.