Dual-Interest Decomposition Head Attention for Sequence Recommendation with Positive and Negative Feedback

Accurate user interest modeling is crucial for recommendation scenarios, especially in short‑video feeds where each exposure is a single video and user actions (skip or not) serve as negative or positive feedback, posing challenges for capturing feedback transition patterns.

To address this, the authors introduce a model composed of a feedback‑aware encoding layer, a dual‑interest disentanglement layer, and a dual‑interest prediction layer. The encoding layer employs a factorized head attention mechanism that masks specific heads and injects feedback information to model the evolution of different feedback types.

In the feedback‑aware encoding stage, multi‑head attention heads are divided into positive‑feedback‑specific and negative‑feedback‑specific groups, allowing each head to capture distinct feedback relationships via a masked matrix that incorporates feedback priors.

The dual‑interest disentanglement layer splits the mixed sequence of positive and negative feedback into two independent sequences, encodes each with a separate factorized attention module, and then disentangles the representations to isolate positive and negative interests.

The dual‑interest prediction layer feeds the separated representations into two independent towers, applying Bayesian pairwise ranking (BPR) loss and contrastive learning to model the pairwise relationship between positive and negative feedback, ensuring the positive tower’s score exceeds the negative tower’s for positive samples and vice‑versa.

Experiments on the Amazon e‑commerce dataset and the Kuaishou short‑video dataset using AUC, GAUC, MRR, and NDCG demonstrate that the proposed model consistently outperforms state‑of‑the‑art sequential recommendation methods, especially on longer interaction histories where the factorized head attention effectively leverages extensive feedback context.

Visualization of attention weights shows interpretable patterns: in short‑video data, negative feedback has limited influence on subsequent items, while in e‑commerce data, negative feedback (e.g., wrong purchases) strongly affects future behavior, confirming the model’s ability to capture feedback‑dependent dynamics.