Mixture of Virtual‑Kernel Experts (MVKE) for Multi‑Objective User Profile Modeling

In the KDD 2022 conference, Tencent Advertising presented a new multi‑task model called Mixture of Virtual‑Kernel Experts (MVKE) for multi‑objective user profile modeling, aiming to improve recommendation and advertising efficiency.

The paper first outlines the importance of diverse and accurate user profiles in advertising and recommendation systems, and describes the limitations of existing dual‑tower models, especially when dealing with sparse data and the need for cross‑action information fusion.

To address these challenges, MVKE abandons the conventional dual‑tower architecture and introduces Virtual‑Kernel Experts (VKE) and a Virtual‑Kernel Gate (VKG). Each VKE focuses on a specific aspect of user preference, while the VKG, based on an attention mechanism, aggregates the outputs of multiple VKEs under the guidance of the label tower, effectively bridging the two towers.

For single‑action scenarios, MVKE retains a twin‑tower‑like structure but adds a “bridge” that enables information flow between the user and label towers. In multi‑action scenarios, multiple VKEs are assigned to different tasks (e.g., CTR and CVR prediction), and the number of VKGs corresponds to the number of tasks, allowing shared and task‑specific experts.

The inference process changes the traditional dual‑tower efficiency: user representations become label‑dependent, increasing space complexity to k·|U| (k = number of VKEs) while keeping time complexity similar, a trade‑off acceptable in practice.

Experimental results show that MVKE outperforms baseline models (no‑MTL, hard‑sharing, MoE, MMoE, CGC) in both offline benchmarks and online A/B tests, delivering significant improvements in business metrics such as GMV and spend across various user groups.

In conclusion, MVKE provides a scalable solution for large‑scale user modeling, handling multi‑topic and multi‑action preferences effectively, and has been successfully applied to improve ad ranking, bidding, and cold‑start scenarios within Tencent’s advertising system.