Explore CBLUE: China’s Premier Biomedical NLP Benchmark and Its 8 Datasets

Developers often face brilliant algorithms without data for validation or lack computational resources to evaluate model scores; the Tianchi platform now offers a ranking feature featuring eight high‑quality medical NLP datasets.

The Chinese Biomedical Language Understanding Evaluation (CBLUE) challenge, launched by the Chinese Information Processing Society’s Medical Health and Bio‑information Committee and hosted on Alibaba Cloud’s Tianchi platform, is co‑organized by institutions such as Yidu Cloud, Ping An Medical Technology, Peking University, Zhengzhou University, Pengcheng Laboratory, Harbin Institute of Technology (Shenzhen), Tongji University, Quark, and Alibaba DAMO Academy to advance Chinese medical NLP.

CBLUE has been online since April 1 2021, with daily rankings updated at 8 am and monthly awards (King, Star, Diamond) based on the final ranking of each month.

As the first Chinese medical information processing challenge, CBLUE covers eight NLP tasks, extending previous CHIP evaluations with real‑world, noisy data that raise the bar for model robustness.

CMeEE

The Chinese Medical Entity Extraction dataset, provided by Peking University, Zhengzhou University, Pengcheng Laboratory, and Harbin Institute of Technology (Shenzhen), is a standard NER task with nine entity types (disease, symptom, drug, equipment, procedure, body, lab test, microorganism, department) and nested entities, making it more complex than typical NER.

CMeIE

The Chinese Medical Information Extraction dataset shares the same providers and focuses on relation extraction with 53 relation types. It requires end‑to‑end extraction without pre‑identified entities and lacks explicit entity offsets, adding difficulty when entities appear multiple times.

Additional annotation includes a “Combined” field indicating whether the two entities appear in the same sentence, highlighting the challenge of cross‑sentence relation extraction.

CHIP‑CDN

The Clinical Diagnosis Normalization dataset, supplied by Yidu Cloud, maps clinical entities to standard ICD‑10 codes, reflecting real‑world diagnosis normalization challenges where diverse medical expressions may correspond to multiple standardized terms.

CHIP‑CTC

The Clinical Trial Criterion dataset from Tongji University presents a multi‑class text classification problem with 44 categories, where class imbalance is a key difficulty for participants.

CHIP‑STS

The Semantic Textual Similarity dataset from Ping An Medical Technology is a binary similarity task across five disease categories, generally considered less challenging.

KUAKE‑QIC

The Query Intention Classification dataset from Quark is an 11‑class text classification task based on real user queries, introducing noise that tests model robustness.

KUAKE‑QTR

The Query‑Title Relevance dataset, also from Quark, is a four‑level relevance matching problem where subtle semantic differences (e.g., “thigh” vs. “leg”) require medical knowledge for accurate scoring.

KUAKE‑QQR

The Query‑Query Relevance dataset, again from Quark, is a three‑level matching task; distinguishing unrelated queries (e.g., different fruits) demands both medical knowledge and common sense.

Overall, the eight tasks span sequence labeling, text classification, and sentence‑level relation judgment, offering a comprehensive benchmark for language understanding in the medical domain and encouraging collaboration between industry and academia.