Big Data Meets Generative AI: Industry Transformations from Prof. Dou

01 Personal Background & Insights on Big Data & AI

Prof. Dou Dejing, a distinguished professor at Fudan University, chief scientist of Beidou Data Intelligence, and part‑time professor at Tsinghua University, founded the Fudan Data Intelligence Lab to advance frontier research and practical applications of big data and artificial intelligence.

02 Development of Artificial Intelligence

The AI timeline starts with the 2010 big‑data surge (from 0.8 ZB in 2009 to 35 ZB in 2020, a 44‑fold increase) and proceeds to the 2022 emergence of large models such as ChatGPT, which grew to 100 million users in five days. AI breakthroughs include AlphaGo (2016) defeating world champion Lee Se‑dol, AlphaGo Zero (2017) learning from scratch, and the evolution of the Turing test toward human‑level interaction.

Big data provides the massive training datasets required for AI models, while AI extracts value from those datasets, creating a mutually reinforcing relationship.

03 Generative AI Breakthroughs

Since the end of 2022, generative AI—exemplified by ChatGPT—has attracted massive attention, driven by the 2017 Transformer architecture and large‑scale pre‑training. Reinforcement Learning from Human Feedback (RLHF) enables models to improve dialogue quality through human‑derived reward signals.

Early dialogue systems relied on database or knowledge‑base queries; later models such as Seq2Seq (2014) and ChatGPT generate context‑aware, multi‑turn conversations, handling up to 4,096 tokens (≈3,072 words).

04 Large‑Model Advantages & Disadvantages

Advantages : Massive parameter counts (e.g., GPT‑3 with 175 billion parameters, GPT‑4 with 1 trillion) produce emergent capabilities, enable complex tasks like natural‑language generation, code synthesis, and reasoning, and reduce the need for labeled data through unsupervised learning.

Disadvantages : Training requires extreme compute resources (thousands of A100 GPUs) and long time, leading to high cost. Even with optimization, large‑model training can span years, though specialized domains (e.g., drug discovery) can achieve rapid results.

05 Emerging Optimized Models

Models such as DeepSeek‑R1 (pure RL training) cut compute usage by ~30 %, while DeepSeek‑MoE (mixture‑of‑experts) improves efficiency. These optimizations promise up to 88 % productivity gains for software developers.

06 Industrial Applications of Large Models

Examples include a social‑security chatbot fine‑tuned on ChatGLM‑6B, achieving near‑human accuracy; business‑script auditing across 700+ datasets with accuracy improvements from 3 to 61 correct detections; and an insurance‑sales assistant that tailors recommendations based on client income.

Generative AI also boosts fraud detection accuracy to 98 % in finance, raises retail conversion rates by 1.5 ×, and cuts insurance‑service costs by 30 %.

07 Future Outlook & Market Potential

Large models will expand into government, advanced manufacturing, transportation, healthcare, media, and education, with specialized models (e.g., DeepSeek) reducing inference cost by >90 % and driving a 4,000‑fold increase in global compute demand by 2030.

The rise of AI agents—ten times more valuable than traditional SaaS—will further accelerate adoption across sectors.

In summary, the talk highlighted the historical synergy of big data and AI, the rapid progress of generative models, their strengths and limitations, and the vast opportunities they create for industry transformation.