Transformer Applications in Meituan Search Ranking: Practice and Experience

Meituan Search connects users and merchants, and ranking is a critical component of the search pipeline. The system uses a multi‑stage ranking architecture (coarse, fine, heterogeneous) with a DNN as the core fine‑ranking model. Recent advances in Transformer‑based NLP models (e.g., BERT) have motivated the exploration of Transformer techniques in search ranking.

Transformer Overview – The Transformer, introduced in the paper “Attention Is All You Need”, adopts an encoder‑decoder structure composed of stacked Multi‑Head Attention and Feed‑Forward Network (FFN) blocks. It replaces recurrent connections with self‑attention, enabling efficient parallel computation.

Feature Engineering – To capture high‑order feature interactions, Meituan adapts the AutoInt approach by inserting Transformer layers after selective embedding of dense and sparse features. Adjustments include (1) retaining an MLP path for dense features, and (2) feeding only a subset of feature embeddings into the Transformer to control model complexity.

Behavior Sequence Modeling – User behavior sequences provide rich signals but are sparse when represented by raw user IDs. Simple sum‑ or mean‑pooling ignores the varying importance of items. Three Transformer‑based models were iteratively developed:

Version 1: Directly feed the raw behavior sequence into a Transformer and apply sum‑pooling on the output.

Version 2: Introduce a target item to allow the user representation to vary across candidate items.

Version 3: Combine the Transformer with an attention‑pooling mechanism (inspired by DIN) to better capture item‑to‑target interactions.

The third version achieved the most significant online improvements in QV_CTR and NDCG.

Re‑ranking – Re‑ranking refines the ordered list produced by the fine‑ranking stage. A sequence‑to‑sequence Transformer model, based on the PRM architecture, was employed. The model consists of feature vector generation, position encoding, a single Transformer encoder layer (Multi‑Head Attention + FFN), and a final scoring head. This design yields stable gains in both NDCG and QV_CTR.

Summary and Outlook – Since late 2019, Meituan has accumulated practical experience with Transformers in ranking, achieving measurable online gains. Future work includes exploring BERT‑enhanced feature layers, richer user data (e.g., graph neural networks), reinforcement‑learning‑driven online re‑ranking, and multi‑objective optimization across diverse business domains.