Pairwise Data Based A/B Experiments: Unbiased Causal Inference in Network Experiments

The DataFun Data Science Summit, held on May 25, invites eight experts to share the latest practices in data science; among them, Tencent data scientist Li Yilin will present a talk titled “Pairwise Data Based A/B Experiments.” Interested participants can register by scanning the QR code and watch the live stream.

Li Yilin is a PhD candidate in Statistics at Peking University, focusing on causal inference with interference, especially in observational data analysis. He works on the WeChat experiment platform and has published in journals such as Biometrics, ACM/IMS Journal of Data Science, and conferences like ICML.

The presentation explains that pairwise data—a distinct data type describing interactions between two entities—enables deeper study of relationships in fields ranging from international trade to social‑network communication. In network‑based A/B testing, the standard SUTVA assumption often fails due to interference, leading to biased global average treatment effect estimates. Li introduces a novel pairwise interference assumption, proves that unbiased global effect estimators generally do not exist for unit‑level outcomes, and designs unbiased estimators that exploit pairwise outcomes. He demonstrates that these estimators remain unbiased under Bernoulli, complete, and cluster randomizations, analyzes their convergence rates, links them to network structure, and establishes asymptotic normality via Stein’s method. Confidence‑interval construction for Bernoulli randomization and related statistical inference methods are also discussed, supported by extensive numerical experiments and a large‑scale online randomized control trial.

Audience takeaways include: (1) an overview of methods for estimating global causal effects in network experiments; (2) a clear understanding of pairwise data analysis; and (3) practical guidance on applying pairwise data to A/B experiments and causal inference, along with the underlying theory and existing challenges.