Multi-Task Learning in Natural Language Processing

Speaker Qiu Xipeng, associate professor at Fudan University, presented a talk titled "Multi‑Task Learning in Natural Language Processing".

The talk was organized into four parts: (1) Deep learning for NLP, (2) Challenges of deep learning in NLP, (3) Multi‑task learning in NLP, and (4) New multi‑task benchmark platforms.

It began with an overview of the lab’s work, including the FudanNLP open‑source system and the upcoming fastNLP toolkit.

Deep learning has become the dominant approach for NLP tasks such as speech recognition, understanding, generation, and human‑computer interaction, with core techniques like machine translation, QA, sentiment analysis, information extraction, summarization, and textual entailment.

Current challenges include limited labeled data and shallow network depths; solutions involve unsupervised pre‑training, multi‑task learning, and transfer learning.

Unsupervised pre‑training methods such as ELMo, OpenAI GPT, and BERT were described, highlighting their shift from word‑level to sentence‑level representations.

Multi‑task learning was explained with examples, historical background (originating in 1997), and its benefits: implicit data augmentation, better representation learning, regularization, and “eavesdropping” where one task can learn features useful for another.

Various sharing architectures were detailed: hard sharing (shared lower layers), soft sharing (cross‑stitch networks), shared‑private, function‑level sharing, hierarchical sharing, and main‑auxiliary task setups.

Search‑based sharing methods that automatically select modules from a shared pool were also presented, illustrating flexible composition of task‑specific pipelines.

A new benchmark platform, the “Ten‑Task All‑Rounder”, converts ten typical NLP tasks into a reading‑comprehension format, and the GLUE benchmark was mentioned as another unified evaluation suite.

The conclusion summarized the covered topics and noted that multi‑task learning often yields higher performance than transfer learning while being less demanding than large‑scale pre‑training.

Future work includes releasing the modular fastNLP toolkit (with interfaces to SpaCy, AllenNLP, AutoML) and its four components: encoder, interaction, aggregation, and decoder.

Contact information and community details for DataFun were provided.