Advances in Dialogue Systems: Baidu PLATO Large‑Scale Conversational Models

Dialogue technology is a core capability for digital human interaction; this presentation focuses on Baidu PLATO research and applications, discussing how large models influence dialogue systems.

1. Dialogue system overview – Task‑oriented systems (e.g., phone assistants) follow a modular pipeline of understanding, management, and generation, while recent end‑to‑end models such as Google Meena, Meta Blender, and Baidu PLATO generate responses directly from context.

2. End‑to‑end generation – Encoder‑decoder architectures (RNN, LSTM, Transformer) encode conversation history into vectors and decode replies, trained on large conversational corpora with a negative log‑likelihood objective.

3. Challenges of open‑domain dialogue – Large pretrained models still produce empty, generic replies and may hallucinate inaccurate knowledge.

4. Baidu PLATO series

• PLATO‑1 introduces discrete latent variables to model the one‑to‑many nature of open‑domain replies, enabling diverse generation via Gumbel‑Softmax sampling.

• PLATO‑2 scales to 1.6 B parameters, uses 12 B Chinese and 7 B English dialogue pairs, and adopts curriculum learning (simple → complex) with a unified PrefixLM architecture for both understanding and generation.

• PLATO‑XL reaches 100 B parameters for bilingual dialogue, achieving superior relevance, richness, and engagement compared with commercial baselines.

5. Knowledge‑enhanced dialogue

PostKS employs posterior‑guided knowledge selection to align prior and posterior knowledge distributions during training. PLATO‑KAG jointly optimizes unsupervised knowledge selection and response generation, encouraging the model to choose useful knowledge. Comprehensive knowledge augmentation (external unstructured knowledge + internalized QA knowledge) reduces knowledge‑error rates from 30 % to 17 % and improves consistency and accuracy.

6. Real‑world deployments – PLATO powers open‑domain chat in smart speakers, virtual humans, community chat, Baidu App’s “Du XiaoXiao” digital person, and the Baidu Input Method virtual assistant.

7. Production challenges

• Inference performance: operator fusion reduces operators by 98 %, cutting latency from 1.2 s on V100 to <300 ms on A10; precision optimization saves 40 % memory, enabling cheaper hardware.

• Dialogue safety: extensive data cleaning, safety classifiers, keyword filtering, and adversarial training mitigate harmful, political, or privacy‑violating outputs.

8. Outlook – Although large models now generate fluent, diverse, cross‑domain conversations, further research is needed on emotion, persona, reasoning, and reducing knowledge misuse, especially as model sizes grow to trillions of parameters.