Industrial Practice of Meta‑Learning and Cross‑Domain Recommendation in Tencent TRS

The talk, titled "Tencent TRS Meta‑Learning and Cross‑Domain Recommendation in Industrial Practice," is divided into two parts: meta‑learning and cross‑domain recommendation, each covering problem definition, solution, and industrial challenges, as well as generalization and deployment practices.

1. Pain points of personalized modeling

In recommendation scenarios, data follows a long‑tail distribution where 20% of scenes cover 80% of samples, causing a single model to favor large scenes. The challenge is to improve model personalization across all scenes.

Industry solutions

PPNet/Poso: uses bias gates for personalization with good performance and cost, but shares parameters across scenes, limiting personalization.

On‑device personalization: deploys a model per device and trains with real‑time data, but depends on device resources and requires small models.

To address these issues, the proposed approach leverages cloud compute to deploy a dedicated model per scene for extreme personalization while maintaining a generic architecture applicable to user, group, or item modeling.

2. Meta‑learning to solve model personalization

The goal is to deploy a personalized model for each user or group without sacrificing cost or performance. By keeping a common model structure and providing scene‑specific parameters, the solution balances training and serving costs.

Deployment is on the cloud, utilizing abundant compute resources and allowing flexible model control.

The algorithmic idea is to apply meta‑learning, traditionally used for few‑shot and cold‑start problems, to achieve extreme personalization in recommendation.

Meta‑learning overview

Meta‑learning learns general knowledge to guide new tasks, enabling rapid adaptation with few samples. For example, after learning addition and subtraction, the model can quickly learn mixed operations like multiplication and division.

Traditional learning optimizes a global parameter θ for all data, favoring large scenes. Meta‑learning optimizes per‑task parameters, allowing fine‑tuning on each scene to achieve optimal performance.

Meta‑learning methods are classified into:

Metric‑based (e.g., Prototypical Networks, Matching Networks)

Model‑based (e.g., Memory‑augmented Neural Networks)

Optimization‑based (e.g., MAML)

3. Meta‑learning algorithm

MAML (Model‑Agnostic Meta‑Learning) consists of meta‑train and fine‑tune phases. In meta‑train, an initialization θ is sampled, then two sampling steps occur: scene sampling and sample sampling within each scene. Each scene provides a support set and a query set; the support set updates θ via SGD, and the query set computes loss, which is aggregated and back‑propagated to update θ . This repeats for multiple rounds.

Fine‑tune mirrors meta‑train: θ is placed in a specific scene, a support set is obtained, and SGD updates the scene‑specific parameters.

4. Industrial challenges of meta‑learning

Applying meta‑learning at scale faces challenges: double sampling requires organizing samples by scene, storing massive dictionaries, and consuming significant storage and compute resources. Sample selection must occur within each meta‑train batch, and training billions of models demands framework modifications.

Proposed solutions include:

Performing sample selection inside the meta‑train batch and using a lightweight framework to support massive meta‑learning training, storing only one set of model parameters and focusing on core network parts.

During serving, fine‑tune using only intermediate‑layer data instead of full raw samples, reducing storage and maintenance costs.

Additional engineering tricks involve storing meta‑learning inputs in a lookup table (dense variables on parameter servers) or using a hash table keyed by scene to reduce network traffic, and applying time‑based eviction to keep the model size manageable.

In serving, embeddings are fed to bottom layers, but meta‑learning layers update parameters using the support set, and the updated parameters are used for scoring. This computation runs on CPU, while batch inference runs on GPU with auto‑batching to stabilize gradients.

5. Industrial practice of meta‑learning

A framework and components abstract meta‑learning, allowing users to only modify model code and provide loss, task inputs, and labels. The generic code supports multiple business scenarios, reducing development effort and resource consumption.

Meta‑learning is applied in dual‑tower recall (user and item towers), sequence recall, and ranking scenarios, achieving significant gains across different contexts.

6. Cross‑domain recommendation

Cross‑domain recommendation faces pain points such as high cost of building separate recall, coarse‑ranking, and fine‑ranking pipelines for each scene, especially for sparse long‑tail traffic. Aligning features and objectives across domains is difficult.

The proposed solution uses shared embeddings for common features and independent embeddings for scene‑specific features. A shared expert processes all data, while each scene has its own expert; a personalization gate fuses shared and scene‑specific outputs before the tower. This architecture supports various model backbones (ShareBottom, MMoE, PLE) and enables knowledge transfer across scenes.

Generalization efforts separate personalized components (highlighted in red) from shared code, which is packaged in ModelZoo and exposed as workflow components, simplifying integration.

Benefits include reduced sample processing cost (‑21%), lower CPU data fetching cost (‑24%), faster iteration (‑40%), and overall reduction in training, serving, and model size, leading to substantial cost savings.

Multiple deployment patterns are described, ranging from single‑objective multi‑scene models to multi‑objective unified models, each with recommendations on when to use star‑type towers.

Overall, meta‑learning and cross‑domain recommendation have delivered notable performance improvements and cost reductions in Tencent's industrial recommendation systems.