Deep Match to Rank (DMR) Model for Personalized Click‑Through Rate Prediction

Background

In e‑commerce recommendation, the task is to select items that a user is most likely to click from a massive candidate pool. Retrieval is typically divided into two stages: Matching (candidate generation) selects a few thousand items using collaborative filtering, and Ranking predicts the click‑through rate (CTR) of these candidates. Personalization is crucial for CTR prediction, and the paper introduces a novel ranking model that incorporates user‑to‑item (U2I) relevance learned from matching, improving model personalization.

User intent in recommendation is implicit and hidden in the user behavior sequence, which can be treated as a query. Prior works such as DIN and DIEN model user interests; this work goes further by modeling U2I relevance directly, which measures the strength of a user's preference for a target item.

U2I relevance can be obtained via collaborative filtering: item‑to‑item (I2I) CF computes similarity between items and infers U2I relevance, while factorization directly computes the inner product of user and item representations (U2I CF). Recent deep learning methods like NAIS and DeepMatch extend these ideas. The authors construct two sub‑networks based on U2I CF and I2I CF to represent U2I relevance.

Model Overview

The DMR (Deep Match to Rank) architecture is shown in the figure. Simple MLP‑based feature crossing cannot capture U2I relevance. Besides handcrafted U2I cross features, the model uses a User‑to‑Item sub‑network and an Item‑to‑Item sub‑network to explicitly model U2I relevance, enhancing expressive power.

User‑to‑Item Network

Inspired by factorization, the inner product of user representation and item representation is used to model U2I relevance, which is an explicit feature crossing. User representation is obtained by average‑pooling the behavior features, weighted by an attention mechanism that uses positional encoding (as in Transformer) as query. The attention weight for the t‑th behavior is denoted α_t, and the weighted sum produces a fixed‑length user vector u ∈ ℝ^{d_v}.

The target item representation is obtained by an embedding lookup from a separate output embedding matrix V′, analogous to the input embedding matrix V used for user representation. The U2I relevance r is then computed as the inner product r = uᵀ·v, where v is the target item embedding.

The loss encourages larger r values for positive interactions, but direct supervision from click labels is insufficient. Therefore, an auxiliary match network (DeepMatch) is introduced, using user behavior as label supervision for the User‑to‑Item network.

Item‑to‑Item Network

The Item‑to‑Item network computes I2I similarity between the target item and each item in the user behavior sequence, then aggregates these similarities via additive attention. This yields another form of U2I relevance, complementing the inner‑product based relevance from the User‑to‑Item network.

Experiments

Offline experiments were conducted on Alibaba’s public dataset and the production dataset of 1688 Recommendation. The DMR model consistently outperformed baseline models such as DIN.

Online A/B testing on the 1688 “Recommended for You” service showed a 5.5% relative CTR lift and a 12.8% DPV lift compared with the previous DIN model, leading to full‑scale deployment.

Results and Outlook

The idea of jointly training matching and ranking originated in April, was refined through experiments, and achieved strong online performance by August. The paper was submitted in September and accepted as an oral presentation at AAAI‑20 in November.

DMR provides a framework for unified matching‑ranking training; the U2I relevance module can be easily plugged into existing CTR models as additional effective features. Future CTR model iterations will continue to build on the DMR framework.

References

[1] Deep Interest Network for Click‑Through Rate Prediction – KDD18 [2] Deep Interest Evolution Network for Click‑Through Rate Prediction – AAAI19 [3] Deep Neural Networks for YouTube Recommendations – ResSys16 [4] Attention Is All You Need – NIPS17 [5] Neural Machine Translation by Jointly Learning to Align and Translate – ICLR15 [6] Distributed Representations of Words and Phrases and their Compositionality – NIPS13 [7] NAIS – Neural Attentive Item Similarity Model for Recommendation – TKDE18