Weak Supervision Machine Learning for Ant Group Business Scenarios: Methods, Experiments, and Applications

The presentation introduces weak supervision learning, highlighting three typical problem types—Incomplete Supervision, Inaccurate Supervision, and Inexact Supervision—based on a 2018 survey by Prof. Zhou Zhihua.

In Ant Group’s scenarios, data often suffer from label scarcity or noise, prompting the use of cross‑scene information, expert rules, or existing models to augment training data.

For sample‑deficient settings, a Direct Learning framework is proposed: pseudo‑effects are first estimated from source‑domain control and treated data, then used to train an effect model, while Distribution Adaptation aligns target‑domain distributions via density‑based reweighting.

To address unreliable pseudo‑effects, an MC‑dropout‑based uncertainty estimator provides a reliability score that guides the learning of the effect model.

Extensive experiments (published at CIKM) demonstrate that the proposed method outperforms baselines under noticeable distribution shifts and remains competitive when shifts are mild, confirming its robustness.

When dealing with multiple noisy label sources, the work introduces a self‑cognitive denoising approach: the model learns instance‑wise noise detection and annotator‑wise quality estimation, enabling weighted fusion of labels.

Two key modules—Self‑cognition and Mutual‑denoising—are built on this theory: Self‑cognition assesses sample reliability and annotator quality, while Mutual‑denoising leverages high‑quality sources to generate pseudo‑labels for weaker ones, moderated by learned weights.

A Selective Knowledge Distillation module further compresses the learned knowledge into a lightweight model for deployment in resource‑constrained settings.

Practical applications include cross‑scene causal effect estimation for risk control, marketing coupon allocation, and user profiling, where noisy or sparse labels are common.

The overall framework, validated on several datasets and accepted at ICML, showcases how weak supervision and cross‑domain techniques can effectively tackle real‑world business challenges.