How IAK Transforms Multi‑Domain Recommendation with Pre‑Training and Fine‑Tuning

Introduction

In real‑world scenarios, users exhibit highly dynamic interests across time, region, and context, making it difficult for traditional recommendation systems to capture these changes. Existing multi‑domain learning methods alleviate the problem but are costly to train and modify in industrial settings.

Proposed Paradigm

The authors propose a unified multi‑domain modeling paradigm that treats the recommendation system as a large model . By applying pre‑training + fine‑tuning techniques, the approach achieves precise, low‑cost recommendations across diverse scenarios without extensive architectural changes. The idea draws inspiration from large language models (LLMs) such as LoRA, which acquire general knowledge through massive pre‑training and adapt to downstream tasks via lightweight fine‑tuning.

Information‑Aware Adaptive Kernel (IAK)

From an information‑bottleneck perspective, the paper introduces the Information‑Aware Adaptive Kernel (IAK) . IAK compresses general knowledge from the pre‑trained model, retains information relevant to downstream tasks, and injects new domain‑specific knowledge. Its core consists of two stages:

Knowledge Compression : Extracts task‑relevant knowledge from the pre‑trained model while discarding irrelevant parts.

Knowledge Matching : Supplements the compressed knowledge with domain data to adapt to specific scenarios.

The process is implemented via Gaussian approximation for parameter optimization, acting as a “smart filter” that remembers useful information and forgets the rest.

Feature Embedding Module

The system ingests six groups of features:

User profile features (e.g., gender, age, income).

Item features (e.g., category, brand, location).

Spatio‑temporal features (e.g., location, time, dwell time).

Recent item statistics (clicks, orders, conversion rates).

Cross features between users and items.

User behavior & coupon sequences.

These diverse representations enable the model to capture fine‑grained user interests across multiple dimensions.

Training and Deployment Strategies

To address industrial constraints, the authors propose:

Parallel Inference & Domain Activation : Deploy all IAK instances simultaneously; at inference time, select the output corresponding to the request’s domain ID.

Dynamic Batch‑Aware Training : Adjust learning rates based on sample counts and gradient magnitudes to stabilize training across heterogeneous tasks.

Experimental Evaluation

Extensive offline experiments on 11 real‑world datasets (covering multiple regions, time slots, and scenarios) show that OLR+IAK consistently outperforms strong baselines (SB, ESMM, MMoE) in both CTR‑AUC and CTCVR‑AUC. Online A/B tests on a billion‑scale platform demonstrate significant lifts in order count, order rate, and net GMV across single‑domain and multi‑domain deployments.

Challenges and Insights

The study discusses pseudo‑cold‑start issues, user/item overlap across domains, and the importance of weighting samples from related domains during training. Sensitivity analysis reveals that an encoder dimension of 50 balances performance and efficiency.

Conclusion

IAK provides a flexible, low‑cost fine‑tuning solution for large‑scale recommendation systems, improving both offline metrics and real‑world business outcomes. Future work includes exploring more effective methods for merging models across different themes.