Document Intelligence in the Financial Sector: Technologies, Challenges, and Future Directions

iWudao Tech combines AI and finance to provide large‑scale financial data services, and in Q2 2024 it reviews the rapid adoption of large‑model AI in the industry.

The talk is organized into five parts: (1) the technical scope of document intelligence, (2) its applications and challenges in the financial domain, (3) document analysis and recognition pipelines, (4) document understanding, and (5) future prospects.

Document intelligence, also called Document AI, covers document analysis (image processing, layout analysis, OCR) and document understanding (semantic extraction, knowledge‑graph construction). Typical financial documents are long, multi‑page PDFs with complex layouts, low‑quality scans, and dense tables, posing challenges for token limits and visual complexity.

Recent advances such as CNN/ViT backbones, Vision Transformers, multimodal fusion, OCR‑Free models (e.g., Donut, StrucTexTv2), and large‑model based layout encoders (LayoutLMv3, LiLT, GeoLayoutLM) are surveyed, with specific experiments on replacing ResNet‑50 with Swin‑T in Mask R‑CNN and integrating multimodal cues to improve detection of text, tables, and graphics.

In document understanding, multimodal pipelines combine OCR text, visual features, and Transformers to perform information extraction, event detection, and chart understanding, illustrated by a joint project on equity‑structure diagram extraction using VSR and Oriented R‑CNN.

The future outlook highlights the need for models that generalize across diverse financial document types, handle hundreds of pages efficiently, and meet strict data‑security requirements, while anticipating that unified large models may eventually replace separate NLP and CV solutions.