DR-BERT: Enhancing BERT-based Document Ranking with Task-adaptive Training and OOV Matching

DR‑BERT boosts BERT‑based document ranking on the MS MARCO benchmark by applying domain‑adaptive pretraining, a two‑stage fine‑tuning pipeline (pointwise then listwise), and OOV‑aware mechanisms—including exact‑match features and word‑recovery of sub‑tokens—achieving the first MRR@10 above 0.4 and leading the leaderboard.

BERTDR-BERTMS MARCO

0 likes · 16 min read

DR-BERT: Enhancing BERT-based Document Ranking with Task-adaptive Training and OOV Matching

Alibaba Cloud Developer

Jul 3, 2019 · Artificial Intelligence

Alibaba AI Sets New Record in MS MARCO Reading Comprehension, Surpassing Humans

Alibaba's AI model topped the MS MARCO reading comprehension challenge, achieving the highest scores in document ranking and open‑domain question answering, even surpassing human performance, thanks to its deep‑cascade BERT‑based architecture that mimics human reading and is already deployed in e‑commerce applications.

BERTMS MARCOReading Comprehension

0 likes · 5 min read

Alibaba AI Sets New Record in MS MARCO Reading Comprehension, Surpassing Humans