Alibaba AI Sets New Record in MS MARCO Reading Comprehension, Surpassing Humans

The MS MARCO reading comprehension challenge, a benchmark with over one million questions and nearly ten million web documents, requires participating AI models to retrieve correct answers from large, multi‑paragraph sources.

Alibaba's AI model achieved the top position in both document ranking and open‑domain question answering, and notably surpassed human performance on the open‑domain QA task, indicating a new level of machine reading comprehension.

Unlike the SQuAD competition, MS MARCO simulates real‑world search engine scenarios, demanding the ability to understand long documents, determine whether an answer exists, and locate the relevant passage, with some questions requiring the model to infer answers not explicitly present.

Alibaba's breakthrough is the "deep‑cascade machine reading model" built on a structured‑information BERT architecture. The model first skims documents quickly, then performs deep reading on selected passages, generating answers based on its own understanding. This approach was accepted at the AAAI 2019 conference.

The technology is already applied in Alibaba's e‑commerce platforms. For example, during a Lazada promotion, the AI learned 25 Indonesian sales rules in 30 ms and answered customer queries with a 96 % accuracy rate.