Ads Recommendation in a Collapsed and Entangled World: Tencent's Innovations in Feature Encoding, Dimensional Collapse Mitigation, and Interest Disentanglement

Recommendation models are the core of computational advertising, and Tencent has upgraded its large‑scale recommendation models to support rapid product evolution. The paper outlines three key research directions: feature encoding, dimensional collapse, and interest entanglement.

Overall Architecture – The single‑task architecture (illustrated in Figure 1) consists of four modules: feature encoding, multi‑embedding lookup, expert networks, and a classification tower. Feature types (sequential, numeric, embedding) are encoded according to their characteristics, then queried from multiple embedding tables before being processed by expert networks and finally fed to the tower.

Feature Encoding

Sequence Features – A Temporal Interest Module (TIM) captures semantic‑temporal relationships between user behaviors and target ads using target‑aware temporal encoding, attention, and representation, achieving superior mutual‑information learning on the Amazon dataset.

Numeric Features – A multi‑radix encoding method converts numeric values into binary/ternary codes, learns embeddings for each digit, and pools them, preserving the inherent order of numeric features. Visual semantic IDs replace raw ad IDs, enabling visual similarity preservation and yielding notable GMV gains.

Embedding Features – Pre‑trained embeddings (e.g., from LLMs or GNNs) are transformed via a Similarity Encoding Embedding (SEE) pipeline that encodes pairwise similarity scores with the same multi‑radix approach, aligning external embeddings with the recommendation model.

Addressing Dimensional Collapse

Embedding dimensional collapse occurs when increasing embedding size does not improve performance because many dimensions shrink to a low‑dimensional subspace. Singular spectrum analysis reveals this phenomenon across feature fields.

To mitigate it, Tencent proposes a Multi‑Embedding Paradigm (ME) that learns multiple embeddings per feature and performs cross‑feature interactions within expert networks, dramatically improving scalability (Figure 6). The GwPFM model further refines this idea by grouping fields and learning part‑aware weights, reducing complexity while retaining effectiveness.

Handling Interest Entanglement

Multi‑task recommendation suffers from shared‑embedding entanglement, where conflicting interests across tasks degrade performance. The Shared and Task‑specific Embedding paradigm (STEM) learns both shared and task‑specific embeddings, coupled with an All‑Forward Task‑specific Backward gating network to isolate gradient updates.

For large numbers of tasks, the Asymmetric Multi‑Embedding Paradigm (AME) decouples the number of embedding tables from task groups, routing tasks to tables of appropriate dimensionality, achieving further GMV improvements.

The STEM‑AL framework extends STEM to auxiliary‑learning scenarios, using separate main and shared embedding tables to boost the primary task while leveraging auxiliary signals.

Summary and Outlook

The article consolidates Tencent's recent advances in advertising recommendation, including feature encoding, dimensional‑collapse mitigation, and interest‑disentanglement techniques, and points to future work on training and analysis tools.

References

1. Ads Recommendation in a Collapsed and Entangled World. KDD 2024. 2. On the Embedding Collapse when Scaling up Recommendation Models. ICML 2024. 3. Temporal Interest Network for User Response Prediction. WWW 2024. 4. STEM: Unleashing the Power of Embeddings for Multi‑task Recommendation. AAAI 2024. 5‑9. Additional related works.