Alibaba Cloud PAI’s Few-Shot NLP Breakthroughs at EMNLP 2022

On December 7, the top international conference EMNLP 2022 opened in Abu Dhabi. EMNLP focuses on empirical research in natural language processing and has driven core innovations such as pre‑trained language models, text mining, dialogue systems, and machine translation. This year, Alibaba Cloud Machine Learning Platform PAI, together with the Alibaba DAMO Academy NLP team and Prof. Gao Ming’s team from East China Normal University, had three few‑shot learning papers selected.

This selection shows that PAI’s self‑developed NLP algorithms and frameworks have reached world‑class performance, gaining recognition from international scholars and demonstrating China’s AI innovation competitiveness.

Few‑Shot Learning Paper Overview

As pre‑trained language models grow larger, fine‑tuning them on downstream tasks typically requires abundant training data to achieve good generalization. Few‑shot learning leverages the knowledge acquired during pre‑training to train high‑accuracy models on very small datasets.

KECP: Knowledge‑Enhanced Contrastive Prompt‑tuning for Few‑shot Extractive Question Answering

Traditional machine reading comprehension relies on large annotated corpora and often overfits in low‑resource settings. Prompt‑tuning mitigates this issue. KECP combines knowledge‑enhanced representations with contrastive learning, converting extractive QA into a generation task based on BERT. Experiments show that with only 16 labeled examples, KECP outperforms previous models.

SpanProto: A Two‑stage Span‑based Prototypical Network for Few‑shot Named Entity Recognition

Named entity recognition traditionally requires substantial data for fine‑tuning. SpanProto addresses data scarcity by decomposing NER into Span Extraction and Mention Classification. The first stage extracts candidate spans without class bias; the second uses prototypical learning to assign labels, handling false positives. Evaluation on the Few‑NERD benchmark demonstrates significant accuracy gains.

UPT: Unified Prompt Tuning for Few‑shot Text Classification

Prompt‑tuning leverages pretrained knowledge for low‑resource tasks. UPT unifies various downstream and pre‑training tasks into a Prompt‑Options‑Verbalizer (POV) format, allowing a single model to solve multiple NLP tasks. By integrating meta‑learning, UPT improves generalization and reduces overfitting. Experiments on GLUE and SuperGLUE datasets show clear accuracy improvements.

EasyNLP Framework and Applications

The source code of the three algorithms will be contributed to EasyNLP, an open‑source Chinese NLP framework built on PyTorch. EasyNLP supports common Chinese pretrained models, large‑model deployment, and provides an end‑to‑end experience from training to serving. It will also support multimodal models, aiming to serve the broader NLP and multimodal research community.

KECP: Knowledge‑Enhanced Contrastive Prompting for Few‑shot Extractive Question Answering

SpanProto: A Two‑stage Span‑based Prototypical Network for Few‑shot Named Entity Recognition

Towards Unified Prompt Tuning for Few‑shot Text Classification