Baidu ERNIE 3.0: Knowledge‑Enhanced 100B‑Parameter Model Sets New Chinese NLP Benchmarks and Tops SuperGLUE

Baidu has launched ERNIE 3.0, a 100‑billion‑parameter knowledge‑enhanced large language model that simultaneously learns from massive unsupervised text and a large‑scale knowledge graph using a parallel Knowledge‑Text Prediction pre‑training method built on PaddlePaddle's distributed training platform.

The model features a two‑layer architecture: a general semantic representation network that captures universal knowledge, and task‑specific semantic networks that inherit from the general layer and can be implemented via auto‑encoding or auto‑regressive structures, enabling both language understanding and generation.

ERNIE 3.0 sets new state‑of‑the‑art results on 54 Chinese NLP tasks—including sentiment analysis, opinion extraction, reading comprehension, summarization, dialogue generation, and mathematical reasoning—often improving performance by more than 3% and achieving the best scores across all evaluated datasets.

Its English counterpart also tops the SuperGLUE benchmark, surpassing Google T5, OpenAI GPT‑3 and even the human baseline by 0.8 percentage points, highlighting its superior commonsense reasoning, causal inference, and coreference resolution abilities.

The model supports zero‑shot learning and fine‑tuning, showing significant gains even with limited labeled data, and can generate literary content such as novels, lyrics, poems, and couplets without task‑specific training.

Both the research paper (https://arxiv.org/pdf/2107.02137.pdf) and an interactive demo (https://wenxin.baidu.com/wenxin/ernie) are publicly available, and the ERNIE series is already deployed in search, information feeds, smart speakers, and Baidu Cloud services across various industries.