Personalized Approximate Pareto-Efficient Recommendation (PAPERec): A Multi‑Objective Reinforcement Learning Framework for User‑Level Objective Personalization

Real‑world recommendation systems often need to optimize multiple objectives such as click‑through rate (CTR), dwell time, diversity, and retention. Existing multi‑objective recommendation (MOR) approaches typically use a single set of objective weights for all users, ignoring individual preferences.

To address this limitation, the authors propose Personalized Approximate Pareto‑Efficient Recommendation (PAPERec), a framework that generates personalized objective weights via a Pareto‑oriented reinforcement learning (RL) module. By approximating Pareto optimality, PAPERec tailors the trade‑off among objectives for each user.

The model builds on the Multiple Gradient Descent Algorithm (MGDA) and its Pareto stationary point theory. The RL module treats the current recommendation list as a state, the generation of user‑specific weights as an action, and defines a reward based on the L2 norm of the weighted sum of multi‑objective gradients, encouraging the system to move toward Pareto‑optimal solutions.

PAPERec is deployed in the WeChat "Look" list‑wise recommendation pipeline, jointly optimizing CTR and dwell time. The overall loss combines single‑objective losses for each metric with the RL‑derived reward. Training uses DDPG, and the architecture incorporates Transformer and GRU components for feature interaction and sequential modeling.

Extensive offline and online experiments on the production system demonstrate that PAPERec achieves the best dwell‑time results while maintaining competitive CTR, outperforming baseline models in the Pareto‑dominance analysis. Further analysis shows that users and items with higher personalized weights for a specific objective indeed exhibit higher performance on that metric, confirming effective objective‑level personalization.

In summary, PAPERec introduces a novel personalized Pareto‑approximation approach for multi‑objective recommendation, combining scalarization, MGDA theory, and reinforcement learning to deliver both academic insights and practical improvements in large‑scale recommender systems.