Fundamentals 3 min read

Systematic Solutions to the AA Problem in Random Experiments

This presentation introduces the AA problem that can compromise A/B test validity and explains how combining re‑randomization with regression adjustment provides an effective, practical solution to improve experiment reliability and credibility.

DataFunTalk
DataFunTalk
DataFunTalk
Systematic Solutions to the AA Problem in Random Experiments

Speaker : Wanbo Kui, Didi Data Analyst. Graduated in June 2021 with a B.Sc. in Statistics and Data Science from Southern University of Science and Technology, and in January 2023 with an M.Sc. in Statistics and Data Science from National University of Singapore. Since January 2023, he works in Didi's data science platform focusing on optimizing all stages of random split experiments.

Talk Title : Systematic Solutions to the AA Problem in Random Experiments

Talk Description : Although A/B testing is considered the gold standard for decision making, its results become questionable when the AA problem exists. Various methods address the AA issue, and the combination of re‑randomization and regression adjustment is among the most effective, helping to mitigate the problem, prevent it proactively, and enhance the trustworthiness of experimental outcomes.

Outline :

1. Survey of re‑randomization research in academia and industry

2. Demonstration of re‑randomization principles and data simulation

3. Practical applications of re‑randomization and key considerations

Audience Benefits :

1. Understand recent advances in covariate balance

2. Familiarize with the underlying mechanisms of re‑randomization

3. Learn how to apply re‑randomization in practice to alleviate the AA problem

Click "Read Original" to view offline conference details.

statisticsA/B Testingexperiment designAA problemre-randomization
DataFunTalk
Written by

DataFunTalk

Dedicated to sharing and discussing big data and AI technology applications, aiming to empower a million data scientists. Regularly hosts live tech talks and curates articles on big data, recommendation/search algorithms, advertising algorithms, NLP, intelligent risk control, autonomous driving, and machine learning/deep learning.

0 followers
Reader feedback

How this landed with the community

login Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.