Learning to Rank (LTR) Practice in Amap Search Suggestions: From Data Collection to Model Optimization

When discussing recommendation systems, most people think of personal recommendation and "thousand‑person‑thousand‑face" (千人千面); the less‑known "thousand‑domain‑thousand‑face" (千域千面) is explored here through Amap's experience.

Application Scenario

Search suggestion (suggest service) automatically completes user queries or POIs during input, listing candidates and ranking them intelligently.

Learning to Rank (LTR) uses machine‑learning methods to solve ranking problems in retrieval systems. Amap adopts GBDT (GBrank) with pair‑wise loss, focusing on sample generation.

Manual labeling of millions of POI candidates is infeasible.

Automatic sample construction from click/no‑click pairs faces click‑over‑fitting, unreliable satisfaction signals, and limited exposure (only top‑10 shown).

Users who type full queries are invisible to the suggestion system, leading to data sparsity.

These issues result in ambiguous modeling when click data is missing, and sparse features are often ignored by the model.

Detailed Solution

Step 1: Merge multiple server logs (suggest, search, navigation) and clean sessions. Step 2: For each session, propagate the click of the final query to all preceding queries, achieving short‑session optimization.

Randomly sample millions of queries, retrieve the top‑N POI candidates, and generate tens of millions of effective samples for GBrank training.

Feature design addresses four modeling needs: comparability across recall chains, dynamic POI demand across queries, supplementing prior features for low‑frequency queries, and regional personalization using geohash partitioning.

Initial model (rules removed) improved MRR by ~5 points, but feature learning was uneven; tree splits favored a few features (e.g., city‑click) while others were ignored.

Analysis identified two main causes: high missing rate of cross‑feature query‑click (≈60% zero) and low discriminative power of text‑similarity features.

Two mitigation strategies were proposed:

Oversample sparse‑feature and low‑frequency query samples (simple but distorts distribution).

Adjust the loss function by adding a penalty term loss_diff to the gradient of the cross‑entropy loss, increasing the split gain for under‑used features. The modified gradient is -$gradient = y_{ij} - 1/(1+exp(-(h(x_i)-h(x_j)) + loss_diff) .

After loss adjustment, MRR gained another 2 points and the coverage of historical ranking cases rose from 40% to 70%.

Specific Effects

LTR replaced rule‑based sorting in Amap's suggestion system, delivering noticeable gains across query frequencies. The model now supports both crowd‑level and individual personalization, and further work includes deep learning, vector indexing, and user‑behavior sequence prediction.

Reference

Machine Learning Optimization Practice in Amap Search Suggestions: https://developer.aliyun.com/article/708264