How Meituan Search 3.0 Leverages LLM Semantic Representations to Boost Ranking

Background & Motivation

Search engines are undergoing a third paradigm shift driven by large language models (LLM): from keyword matching (1.0) to behavior‑based personalization (2.0) and now to complex intent understanding (3.0). In Meituan’s service‑retail search, traditional ranking relies heavily on textual matching, which struggles with long‑tail categories, non‑standardized offers, and ambiguous queries such as “pet SPA + wash”. The team therefore explored using LLM‑generated semantic vectors for query, POI (store) and Deal (product) representations and injecting cosine similarity signals into the ranking model.

Phase 1 – Feasibility Validation

Core Idea

The goal was to verify whether LLM embeddings could materially improve the fine‑ranking model. A lightweight open‑source LLM was fine‑tuned to produce three special tokens <|query|>, <|item|> and <|qi|> that aggregate query, store and joint information respectively. During training, three independent forward passes with attention masks ensured that the <|query|> token only sees the query text, <|item|> only sees the store description, and <|qi|> sees the full sequence for auxiliary loss.

Technical Solution

Training data comprised ~30 million logs (order:click:impression = 1:3:6). Two losses were combined:

Loss₁: cosine similarity between <|query|> and <|item|> embeddings, scaled by a learnable temperature and optimized with binary cross‑entropy.

Loss₂: a click‑rate prediction loss using the <|qi|> token.

Embedding extraction used the last‑layer hidden state of each special token, followed by a two‑layer MLP (hidden → 512 → ReLU → LayerNorm → 64) to obtain 64‑dimensional vectors.

Online Results

In a 20 % traffic A/B test (Sep 18 – Oct 1 2025), the semantic feature increased overall search‑payment orders by +0.20 % and service‑retail orders by +0.27 %. Long‑tail NDCG@5 rose +2.21 pp while BadCase@1 dropped –2.96 pp, confirming that a single 64‑dimensional semantic signal can deliver measurable gains.

Limitations

Phase 1 covered only query‑store pairs, ignored product semantics, required full‑parameter fine‑tuning (high cost), optimized solely for click‑rate (not order), and suffered from inefficient three‑pass inference.

Phase 2 – Systematic Upgrade

Motivation

To address Phase 1’s shortcomings, Phase 2 expanded the training unit to a five‑tuple (query, positive deal, positive POI, hard‑negative deal, hard‑negative POI) and introduced product semantics. Hard negatives were sampled from the same request but unclicked items, ensuring high contextual similarity.

Prompt Design

Experiments compared four prompt styles; the simplest “concise information + summary cue” outperformed more complex task‑oriented prompts, indicating that embedding‑oriented prompts should focus on the information to be aggregated rather than instruction following.

Model & Fine‑Tuning

Multiple base models (0.5 B – 8 B parameters) were benchmarked; a mid‑size embedding‑optimized model achieved the best trade‑off between effectiveness (Click‑AUC, NDCG) and inference cost. Fine‑tuning switched from full‑parameter to LoRA (r = 8, α = 32 on q_proj and v_proj), which improved NDCG while reducing training overhead.

Representation Extraction

Instead of three special tokens in a single sequence, each entity now uses an independent sequence with a learnable “aggregation vector” (nn.Parameter) placed at the last valid token position. This eliminates the need for attention masks, enables a single forward pass for all five inputs, and speeds up inference.

Dimensionality Reduction

Phase 2 replaced the fixed two‑layer MLP with Matryoshka Representation Learning (MRL‑E) using nested dimensions [1024, 512, 256, 128]; at inference the first 128 dimensions are kept, providing multi‑scale flexibility without significant accuracy loss.

Loss Functions

The classification loss was replaced by contrastive learning:

InfoNCE losses for Query↔POI, Query↔Deal, POI↔Deal.

Triplet loss (margin = 0.5) on hard‑negative pairs to sharpen discrimination.

Combined, these losses align the embedding space with the ranking objective (relative ordering rather than binary match).

Feature Integration

Cosine similarity scores were bucketed into ten discrete bins (boundaries: [-0.40, -0.30, -0.18, -0.12, 0.00, 0.10, 0.16, 0.22, 0.30]) and each bin received a learnable 12‑dimensional embedding, which was concatenated with existing ranking features. This discretization improves interaction with the model’s feature‑cross network while reducing noise sensitivity.

Online Results

In a 20 % traffic A/B test (Mar 18‑23 2026), overall search UV rose +0.07 %, effective click QV +0.13 %, and service‑retail result‑page QV‑CTR +0.10 pp. Latency increased only 0.2 ms (TP90). Detailed analysis showed that while overall exposure slightly fell, conversion efficiency improved because more relevant items replaced low‑value exposures.

Performance Gains

Compared with Phase 1, Phase 2 delivered +6.25 pp Q2I‑Click‑AUC, +5.37 pp Q2I‑Order‑AUC, +1.77 pp Q2I‑Click‑NDCG and +2.03 pp Q2I‑Order‑NDCG.

Phase 3 – Transfer to Down‑stream Item Ranking

Motivation

While Phase 2 built a robust representation for store ranking, the downstream “down‑hook” ranking (store → product) still lacked semantic modeling. Phase 3 aimed to reuse the Phase 2 model for product ranking and to fill missing cross‑features (user × product, query × product).

Coverage Challenge

Direct transfer yielded only 81.24 % query coverage and 73.61 % joint query‑product coverage because product‑centric queries differ from store‑centric ones. Re‑selecting the embedding generation scope for the down‑hook scenario raised query coverage to 98.92 % and joint coverage to 89.81 %.

Feature Validation

Embedding cosine scores were bucketed into 100 bins; average click, order and conversion rates increased monotonically with higher cosine, confirming the transferred embeddings remained predictive.

Bucket Strategies

Five bucket schemes were evaluated (equal‑sample, label‑driven equal‑interval for order, shelf‑order, and exposure). The equal‑sample scheme (V1) gave the highest overall importance, while label‑driven schemes contributed complementary gains for specific metrics.

Injection Methods

Four integration approaches were compared:

Bottom‑layer concatenation (+7 bp).

Bottom‑layer + LLM similarity (+18 bp).

Top‑layer similarity before the output tower (+8 bp, some negative effects).

PEPNet gating (+25 bp).

PEPNet gating treats the cosine similarity as a gate that modulates the weight of other features, yielding the strongest offline lift (ctcvr_auc_global_poi + 25 bp).

Global Cross‑Statistics

Four new cross‑statistics were added (user × deal, POI × cate3, user × POI × cate3, query × deal). Low‑coverage order‑related statistics were dropped after ablation.

Online Results

In a 10 % traffic A/B test (May 28 – Jun 3 2026), service‑retail orders increased +0.32 %, visit‑to‑purchase rate +0.29 %, and overall search payment orders +0.35 %. The uplift stemmed from the additive effect of semantic features (+0.20 % from Phase 2) and cross‑statistics (+0.12 %).

Key Insights & Lessons

Mid‑size LLMs (≈1‑3 B parameters) provide the best effectiveness‑cost balance for embedding tasks; larger models give diminishing returns.

Hard‑negative samples dramatically improve discriminative power; “same request, same store, unclicked product” negatives are especially effective for ranking.

Embedding‑oriented prompts should be concise; excessive instruction harms semantic aggregation.

Embedding coverage is the primary risk when transferring models across modules; always verify and re‑select the embedding scope.

Semantic similarity features and traditional statistical features are complementary; combined they yield larger gains than either alone.

Comparison with Prior Work

The article positions its contributions against three research dimensions:

Text‑embedding models (e.g., E5, BGE, GTE) – the work adopts contrastive learning with hard negatives and MRL‑E for multi‑scale embeddings.

Embedding consumption in ranking – unlike recommendation‑focused bottom‑layer concatenation, this work injects cosine similarity both as a discretized bucket (bottom) and as a raw continuous signal (top), and further refines it with PEPNet gating.

Hard‑negative strategies – the “same request, same store, unclicked” scheme provides context‑aware hard negatives, differing from global ANN‑based approaches such as ANCE.

Novelty lies in (1) direct cosine‑similarity feature injection for search ranking, (2) three‑entity (query‑POI‑Deal) contrastive learning, and (3) systematic transfer of embeddings across hierarchical ranking stages.

Future Directions

Improve negative‑sample quality (false‑negative masking, focal re‑weighting, harder mining).

Diagnose low‑dimensional MRL‑E tiers and explore Semantic ID quantization for ultra‑compact representations.

Train a dedicated representation model for the down‑hook scenario to further boost coverage and quality.

Close the producer‑consumer loop by feeding downstream ranking signals back into embedding training (e.g., relevance‑based distillation).

Conclusion

Meituan Search 3.0’s three‑phase practice demonstrates a clear evolution: from proving that LLM embeddings can help ranking, to building a scalable, contrastive‑learning‑driven representation pipeline, and finally to transferring that pipeline across hierarchical ranking stages. Each phase’s technical decisions directly addressed the limitations of the previous one, resulting in consistent offline and online improvements in click, order and NDCG metrics while maintaining low latency.

prompt = """User query: {} <|query|>

Candidate store info:
Store name: {}
Hot‑selling product: {}
Brand: {}
Category hierarchy: {} - {} - {}
User rating: {} points
Average price: {} yuan
Business district: {} <|item|>

Please judge whether the store matches the query <|qi|>"""