Advances in Click‑Through Rate Prediction: Model Evolution, Feature Interaction, Continuous Feature Embedding, and Distributed Training

The presentation begins with a historical overview of recommendation models, tracing their evolution from 2006 collaborative‑filtering and matrix‑factorization techniques, through generalized linear models and factorization machines, to deep learning architectures such as FNN, PNN, DIN, Wide&Deep, and DeepFM, and finally to reinforcement‑learning‑based recommenders.

It then identifies click‑through‑rate (CTR) prediction as the central problem in recommendation systems, explaining how accurate CTR estimation drives revenue and user experience, and categorizes modern CTR models into three groups: combinatorial feature mining, user‑behavior modeling, and automated architecture search.

The article examines user‑behavior modeling advances (e.g., DIN, DIEN, BST) that incorporate pooling, RNNs, or Transformers to capture sequential patterns, and discusses combinatorial feature‑interaction methods ranging from explicit cross features (Wide&Deep) to factorized approaches (IPNN, DCN) and neural interaction networks (PNN).

For continuous‑feature handling, it surveys three strategies—no embedding, field embedding, and discretization—and introduces Huawei's AutoDis framework, which learns meta‑embeddings and automatic discretization to generate high‑quality continuous‑feature embeddings, showing consistent gains on public and private CTR datasets.

The discussion then shifts to large‑embedding training challenges, describing why embedding tables can reach hundreds of gigabytes, and reviews three parallel‑training paradigms: data parallelism (All‑Reduce), model‑parallel embedding sharding, and CPU‑resident embeddings with GPU‑resident MLPs.

To address these issues, Huawei proposes the ScaleFreeCTR system, comprising a host manager for embedding storage and caching, a data loader, and GPU workers that fetch cached embeddings, perform forward/backward passes, and update parameters, thereby improving resource utilization and throughput.

Finally, the talk summarizes three promising research directions: data‑aware model design, accelerating training and data utilization, and increasing automation in data processing, feature selection, and hyper‑parameter tuning to free practitioners for higher‑level innovation.