Baidu Document Intelligence Technology Overview and Applications

Document intelligence technologies are increasingly applied in finance, insurance, energy, logistics, healthcare and other sectors for tasks such as key‑information extraction, document parsing, and document comparison; this article shares Baidu's advances and applications in this field.

The content is organized into five parts: an introduction to document intelligence, the ERNIE‑Layout multimodal large model, the open‑domain extraction‑question‑answer model DocPrompt, layout and table understanding techniques, and the open‑source PaddleNLP ecosystem with practical case studies.

Document intelligence aims to automatically read, understand, and analyze diverse electronic documents (e.g., resumes, invoices, contracts, legal judgments). Major challenges include heterogeneous formats, rich layouts, multimodal information (text, layout, tables, images), and limited labeled data; Baidu addresses these with integrated parsing pipelines, cross‑modal pre‑training, and multi‑stage multi‑task training to enable zero‑shot and few‑shot capabilities.

ERNIE‑Layout combines textual and visual features through a dual‑branch encoder, introduces novel pre‑training tasks such as visual‑language masking (MLVM), token‑in‑image alignment (TIA), reading‑order prediction (ROP), and region‑replacement prediction (RRP), and supports 96 languages, achieving state‑of‑the‑art results on 11 document‑intelligence benchmarks.

DocPrompt adopts a prompt‑based paradigm for open‑domain document extraction and QA, eliminating fixed schemas and enabling zero‑sample extraction across various document types and languages; examples demonstrate its ability to handle spatial reasoning, multi‑dimensional tables, rich visual content, web layouts, long medical texts, and multilingual receipts.

Layout and table understanding follows a three‑step pipeline—layout element detection, reading‑order reconstruction, and element‑relation modeling—leveraging ERNIE‑Layout to unify multimodal tasks and achieving SOTA performance on datasets such as PubTables1M, PubLayNet, and DocLayNet.

PaddleNLP provides open‑source access to DocPrompt and ERNIE‑Layout models, along with task‑flow APIs; application cases include an intelligent customs system, generic contract comparison, and contract review, illustrating how Baidu's document intelligence can be rapidly deployed in industry.

The presentation concludes with acknowledgments and thanks to the audience.