Why FM and FFM Still Dominate Large‑Scale Sparse CTR Prediction

Background

In computational advertising, click‑through rate (CTR) and conversion rate (CVR) are critical metrics; accurate prediction directly boosts ad revenue. Common approaches include Logistic Regression, GBDT + LR, Factorization Machine (FM), and Field‑aware Factorization Machine (FFM). FM and FFM have repeatedly won major CTR competitions such as Criteo and Avazu, demonstrating strong performance on large, sparse datasets.

FM Principle

FM models pairwise feature interactions by representing each feature with a low‑dimensional latent vector. The interaction score is obtained via the inner product of two vectors, allowing the model to capture hidden relationships even when data is extremely sparse, while keeping computational cost linear in the number of non‑zero features.

FFM Improvements

FFM extends FM by assigning a separate latent vector for each feature‑field pair, where a "field" groups related features (e.g., user, item, context). This field‑aware representation enables more precise modeling of interactions across different feature groups, leading to higher predictive accuracy in CTR/CVR tasks.

Implementation Details

Practical deployment involves several steps:

Data preprocessing and feature hashing to manage billions of sparse features.

Generating field‑aware latent vectors, often using open‑source libraries such as libffm or custom implementations.

Training with stochastic gradient descent (SGD) or adaptive variants, applying L2 regularization and learning‑rate schedules.

Hyper‑parameter tuning (latent dimension, learning rate, number of epochs) based on validation AUC.

The Meituan‑Dianping team customized the pipeline for their Demand‑Side Platform (DSP), integrating feature pipelines, model training, and online inference within their ad‑bidding system.

Application in DSP

FFM was deployed to predict both CTR and CVR for real‑time bidding decisions. The model’s superior ranking ability resulted in higher predicted click probabilities, improved bid pricing, and ultimately increased revenue. Empirical results reported a noticeable lift in AUC and revenue per mille (RPM) compared with baseline LR/GBDT solutions.