Improving Search Relevance in PointCheck

Background

Search relevance measures the match between a user query and a merchant (POI) and is critical for the Dianping app. Queries can involve merchant names, dishes, addresses, categories, or complex combinations, while POIs contain many fields (name, address, tags, dishes, etc.), leading to three main relevance problems: textual mismatch, semantic drift, and category mismatch.

Existing Relevance Techniques

Early methods relied on term‑based matching (TF‑IDF, BM25) which are fast but lack semantic understanding. Traditional semantic models include latent space approaches such as Partial Least Squares [1] and translation‑based models [2]. Deep semantic matching now uses representation‑based (e.g., DSSM [3], NRM [4], SentenceBERT [5]) and interaction‑based (e.g., ESIM [6], CEDR [8]) architectures, often built on pretrained language models.

Dianping Search Relevance Design

To address the limitations of pure representation models, Dianping adopts an interaction‑based approach built on the MT‑BERT pretrained model [11]. Three challenges are tackled:

POI Input Construction : Because POIs lack a single high‑information field like a web title, Dianping extracts a POI matching‑field summary that selects the most query‑relevant textual fields in real time (see Fig. 4).

Model Adaptation : A two‑stage training pipeline is used. Stage 1 performs continual domain‑adaptive pre‑training on click and negative‑sample data; Stage 2 fine‑tunes on manually annotated relevance data, enriched with hard‑case mining and contrastive sample generation (Fig. 5‑6).

Online Performance : Since a 12‑layer BERT model is computationally heavy, Dianping introduces caching of high‑frequency query scores, TF‑Serving deployment with the ART framework (based on Faster‑Transformer [15]), a pre‑ranking “golden rule” layer, and parallel execution with the core ranking module (Fig. 8‑9).

POI Matching‑Field Summary

The pipeline computes similarity features between the query and each POI field, selects the most informative fields, and concatenates them with the merchant name and category to form the POI side input.

Two‑Stage Training

Stage 1 uses click data to capture search‑specific semantics (e.g., “开心烧烤” vs. “高兴烧烤” are different brands). Positive samples are queries with click‑through rate above a threshold; negatives are selected via a Skip‑Above strategy and further filtered by rule‑based noise reduction. Stage 2 adds manually labeled data, focusing on hard cases such as cross‑dish matches and edge‑sample mining, and generates contrastive pairs to improve synonym generalization.

Multi‑Similarity Matrix Deep Interaction Model

Inspired by CEDR [8] and MatchPyramid [13], the model splits the encoded query and POI vectors, computes four similarity matrices (Indicator, Dot‑product, Cosine, Euclidean), fuses them, and feeds the result to a scoring layer. The Indicator matrix explicitly encodes token‑level matches (e.g., exact, split, or partial matches) to help the model distinguish high‑textual‑overlap but unrelated cases.

Practical Results

Offline Evaluation

Benchmarking on a manually labeled set shows that the base method (BERT with query + POI summary) improves negative‑example F1 by 1.84 %. Adding two‑stage training raises F1 by an additional 10.35 %, and the multi‑similarity matrix model adds another 11.14 % gain, achieving AUC = 0.96 and overall F1 = 0.97 (Table 1).

Online Impact

After deployment, the monthly BadCase rate on the first‑page results dropped by 2.9 percentage points, and NDCG increased by 2 pp. Example cases (Fig. 10) illustrate how the model correctly promotes “佩姐名品” while demoting unrelated merchants, and distinguishes “榴莲蛋糕” from a non‑relevant “榴莲千层” restaurant.

Conclusion and Outlook

The system combines POI field summarization, domain‑adaptive two‑stage training, and a multi‑similarity deep interaction architecture to achieve strong relevance performance while meeting real‑time inference constraints. Future work includes incorporating richer entity‑type priors, external knowledge, finer‑grained ranking tiers, and extending relevance modeling to non‑merchant search modules.