2020 NLP Milestones & Future Trends: Insights from JD’s AI Scientist

Natural language processing (NLP) has experienced explosive growth, highlighted by the release of GPT‑3 with 175 billion parameters and the rapid maturation of knowledge graphs, multimodal dialogue, and large‑scale pretrained models.

Wu Youzheng , JD Technology algorithm scientist and senior technical director, shared his perspective on NLP developments in 2020 and expectations for the next year during an InfoQ interview.

He noted that GPT‑3 and other pretrained models have attracted widespread attention across AI, with GPT‑3‑generated text often indistinguishable from human writing. Human‑machine dialogue progressed significantly thanks to models such as Google’s Meena, Facebook’s BlenderBot, and the “date” experiment between BlenderBot and Pandora Kuki.

Wu identified several dimensions for reviewing NLP progress: discipline development, technology trends, talent landscape, and real‑world applications. He traced NLP’s evolution from a niche field before 2000 to a cornerstone of search, recommendation, advertising, and social media after the rise of Google.

Key research trends in 2020 include:

Multimodal human‑machine dialogue as the hottest topic, covering task‑oriented dialogue, open‑domain chat, conversational recommendation, and machine reading comprehension.

Continued enthusiasm for text generation, especially controllable generation that integrates multimodal and knowledge signals.

Machine translation remains important but its relative attention has declined.

Traditional tasks such as summarization, syntactic parsing, relation extraction, and named‑entity recognition are gradually fading.

From a model perspective, Wu highlighted:

Pre‑trained encoder‑decoder models (e.g., BERT, GPT, T5) dominate both understanding and generation benchmarks.

Knowledge‑enhanced pre‑training is becoming a universal performance booster, with increasing participation in knowledge‑graph conferences.

Graph neural networks are widely adopted for text classification, relation extraction, multi‑hop reading, and numeric reasoning.

Multimodal processing has permeated many domains, from content generation to dialogue.

Regarding pre‑training trends, he outlined:

Shift from static word embeddings to contextualized pre‑training (e.g., ELMo).

Unification of pre‑training and downstream fine‑tuning (BERT/GPT style).

Encoder‑decoder frameworks (e.g., T5) that handle both understanding and generation.

Multimodal pre‑training on text‑image and text‑video pairs (e.g., VL‑BERT, Unicoder‑VL).

Knowledge‑enhanced pre‑training (ERINE, REALM, K‑BERT, K‑Adapter).

Scaling models to massive sizes (GPT‑3, Switch‑Transformer, Chinese 217 B‑parameter models).

Developing compact models (TinyBERT, ALBERT) for latency‑critical applications.

In practice, JD applies these advances through domain‑adapted pre‑training, knowledge‑enhanced models for product description generation, and model compression techniques such as knowledge distillation to meet latency and cost constraints.

He also discussed the growing importance of multimodal dialogue, noting that over 16 % of JD’s online customer‑service conversations contain images, requiring multimodal semantic understanding.

On knowledge graphs, Wu described two major efforts: a product‑centric knowledge graph for e‑commerce and a pharmaceutical knowledge graph for prescription review, illustrating applications in QA, content generation fidelity, intelligent procurement, and medical prescription auditing.

Wu identified mature NLP areas (text understanding, search, recommendation, translation, dialogue) and rapidly evolving fields (digital content generation, knowledge graphs, multimodal interaction, virtual digital humans), driven by deep‑learning breakthroughs and strong market demand.

He highlighted three major challenges for NLP: ambiguity, diversity, and knowledge representation/integration, as well as the need for symbolic‑neural hybrid reasoning and explainability.

Looking ahead, Wu expects continued focus on pre‑training (larger, smaller, more efficient models), tighter integration of knowledge and data, and multimodal human‑machine interaction that supports rich auditory, visual, emotional, and contextual cues.

JD is already investing in these directions, including a national‑level project on multimodal human‑computer interaction for intelligent customer service, marketing, and media scenarios.