Citation Intent Recognition: Meituan's Winning Solution in WSDM Cup 2020

At the WSDM 2020 conference, Microsoft Research launched the Citation Intent Recognition task, a classic text retrieval ranking challenge that asks participants to retrieve the top‑3 most relevant papers for a given scientific description.

Meituan's Search & NLP team, together with two universities, won the first place in Task 1 by proposing a multimodal retrieval‑ranking framework based on BERT and LightGBM.

1. Background The 13th International Conference on Web Search and Data Mining (WSDM) is a high‑impact venue with a paper acceptance rate of about 15%. The WSDM Cup provides real‑world industrial data for research. In 2020, the Cup featured three tasks; the Citation Intent Recognition task attracted nearly 600 participants worldwide.

2. Task Description Given a textual description of a research contribution, participants must retrieve the three most relevant papers from a large candidate set. The task is essentially a retrieval‑ranking problem evaluated by Mean Average Precision at 3 (MAP@3).

2.1 Evaluation Metric MAP@3 computes the average precision of the top‑3 retrieved documents for each query.

3. Model Methodology

3.1 Retrieval (Recall) Stage Efficient recall is achieved by combining several strategies: a semantic vector model, a spatial vector model with Bag‑of‑Ngram, and axiomatic retrieval models (F1EXP, F2EXP). An inverted index is built, and the top‑50 results from F1EXP, BM25, and TFIDF are merged, achieving ~70% recall on the validation set.

3.2 Ranking (Precision) Stage

Two ranking solutions were explored:

BERT‑based ranking : A pairwise learning‑to‑rank approach replaces the original pointwise formulation. Training samples are constructed as (query, doc1, doc2) triples where doc1 is more relevant than doc2. The loss changes from cross‑entropy to hinge loss. Domain‑specific pretrained models (SciBERT, BioBERT) are used to handle biomedical terminology.

LightGBM‑based ranking : Features include axiomatic scores (F1EXP, F2EXP, BM25, TFIDF), distributed semantic vectors (GloVe, Doc2vec), and ranking‑specific features from the recall stage. LightGBM’s GOSS, EFB, histogram‑based split, and leaf‑wise growth improve efficiency and accuracy.

4. Experimental Results Both BERT and LightGBM pairwise models outperform their pointwise counterparts. Detailed results are shown in Table 2 of the original article.

5. Conclusion and Future Work The combined recall‑plus‑ranking pipeline achieved the top rank in the competition. Future improvements include increasing recall beyond 70% and exploring listwise ranking methods.

6. Practical Applications The solution is being tested in Meituan’s intelligent QA, FAQ recommendation, and core search ranking systems.