Multiple Boosting Calibration Trees (MBCT): Feature‑Aware Binning for Uncertainty Calibration

This document presents a research work on uncertainty calibration for click‑through‑rate (CTR) prediction in advertising systems. The authors introduce a tree‑based, feature‑aware binning framework called Multiple Boosting Calibration Trees (MBCT) and propose a new multi‑view calibration error metric (MVCE) to evaluate calibration quality.

Background. In probabilistic prediction tasks such as CTR estimation, the predicted probability must be well calibrated because it directly influences ranking and billing. Traditional models (logistic regression, neural networks) often produce biased probability estimates, and the true probability is unavailable, making calibration challenging.

Calibration error measurement. The paper defines True Calibration Error (TCE) as a norm‑based distance between predicted and true probabilities. Since TCE cannot be computed in practice, Expected Calibration Error (ECE) is used as an approximation. The authors critique existing ECE variants and introduce MVCE, which averages ECE over multiple random partitionings of the data, thereby providing a more robust, multi‑dimensional assessment.

Calibration methods. Existing methods are categorized into parametric (e.g., Platt scaling, temperature scaling), non‑parametric (e.g., histogram binning), and hybrid approaches. The authors propose a hybrid method that incorporates a feature‑aware binning strategy: a machine‑learning model learns the bias pattern of the original predictor in feature space and groups samples with similar bias into the same bin.

Feature‑aware binning. By capturing bias patterns (e.g., over‑estimation vs. under‑estimation) the method enables personalized, non‑monotonic calibration. MVCE is used as the loss function to guide the learning of binning decisions.

Multiple Boosting Calibration Tree (MBCT). MBCT builds an ensemble of decision trees where each node splits on the feature that yields the greatest MVCE reduction. Leaves contain calibrated outputs derived from a simple scaling function. Boosting is applied to focus on samples with high calibration error. The algorithm supports both per‑sample personalized calibration and non‑order‑preserving adjustments.

Experiments. Offline experiments on the proprietary CACTRDC dataset and public datasets (Porto Seguro, Avazu) show that MBCT consistently achieves lower MVCE than baselines such as Platt scaling, Beta calibration, histogram binning, isotonic regression, and scaling‑binning. Online A/B testing in Alibaba’s advertising platform over 15 days demonstrates significant improvements in click‑through rate and revenue when MBCT replaces isotonic regression.

Conclusion and future work. MBCT advances calibration by integrating feature‑aware binning with boosting, breaking the monotonicity constraint of many existing methods, and delivering both calibration and ranking gains. Future directions include global optimization of the binning process, richer scaling distributions, and calibration tailored to bid*CTR (eCPM) ranking objectives.