This article explores essential sampling strategies for big‑data environments—including simple random, reservoir, stratified, oversampling, undersampling, and weighted sampling—detailing their principles, algorithmic steps, advantages, drawbacks, and suitable application scenarios to help analysts choose the right method.
This article explains why variance reduction is essential for A/B experiments, describes at‑assignment and post‑assignment techniques such as stratified sampling, post‑stratification and CUPED, compares their effectiveness, and presents real‑world case studies demonstrating how they improve experiment sensitivity without increasing sample size.
The paper presents the ‘Post‑Darwin’ evaluation framework, which uses stratified sampling to compare participants and non‑participants across uniform payment layers, overcoming uneven user distributions and the lack of viable A/B tests in game‑business effect analysis, and demonstrates its effectiveness through real‑world promotional and reservation case studies.
The article explains offline sampling for AB testing, detailing why it is needed, the main challenges of limited sample size, population heterogeneity, and non‑random interventions, and presents variance‑reduction, stratified sampling, IPW, and matching methods to address these issues.
Offline sampling in A/B testing assigns experimental units such as users or tags before a trial begins, but suffers from limited sample size, high heterogeneity, and non‑random allocation, which can be mitigated by variance‑reduction methods like CUPED, stratified sampling with inverse‑probability weighting, and matching approaches including propensity‑score matching.
