Calibration4CVR: Representation Calibration for Multi‑Task Learning in Conversion Rate Prediction

Calibration4CVR, a 2018 work on conversion‑rate (CVR) prediction, revisits the development of CVR models from ESMM, ESMM2, MMOE, PLE to NCS4CVR, highlighting two trends: finer‑grained sharing (from layer‑level to neuron‑level) and increasingly automated handling of task conflicts.

Multi‑task learning (MTL) addresses data sparsity by sharing representations across tasks such as click‑through‑rate (CTR) and CVR. In recommendation systems, the key challenges are determining what to share and mitigating negative knowledge transfer when tasks conflict.

The proposed architecture uses representation calibration: shared embedding layers for CTR and CVR, followed by task‑specific MLPs that are calibrated using learned weights and scaling factors. Four calibration variants are described:

CTR calibration with two MLP layers (non‑linear transformation and feature selection) whose outputs are concatenated with CVR features.

CTR calibration with a single MLP followed by a learned scaling factor (0–1) before concatenation.

CVR calibration where a squeeze‑and‑excitation module derived from CTR produces a calibration vector (via sigmoid) that multiplies CVR outputs.

CVR calibration concat that adds the calibrated CVR representation to the original CVR features.

Each variant is illustrated with diagrams (Figures 1‑6) and includes design details such as extra learned weights, feature transformation, and activation functions (sigmoid, ReLU, L1 regularization).

Experiments on large‑scale online data (day‑level training, one‑day testing) show that the calibration layers improve offline AUC by ~0.01 and increase online CVR by about 2.8 % relative to the baseline. The CTR‑MLP variant and the CVR‑Calibration‑Concat variant perform best, while the latter does not significantly surpass the simple CVR calibration, suggesting the model already learns optimal residual information.

Further analysis of CTR/CVR sample ratios indicates a 1:1 training ratio yields the best results, and weakening CVR’s influence on embeddings degrades performance, confirming distinct embedding needs for the two tasks.

Future work includes more thorough benchmarking, clustering analysis of task‑specific embeddings, triplet‑wise loss modeling to alleviate task conflicts, and extending calibration to additional tasks for cumulative knowledge accumulation.

References: Xiao Ma et al., ESMM (SIGIR‑2018); Wen et al., ESMM2 (SIGIR‑2020); Zhao et al., MMOE (RecSys‑2019); Tang et al., PLE (RecSys‑2020); Xiao et al., NCS4CVR (arXiv‑2020).