statistical significance

vivo Internet Technology

Jun 29, 2022 · Operations

Intelligent Gray Release Data Analysis System Practice for Game Center

The article details vivo Game Center’s end‑to‑end intelligent gray‑release data analysis system, which combines experimental design, statistical significance testing, multi‑dimensional anomaly root‑cause analysis (via the Adtributor algorithm), automated reporting, and sample‑size estimation to enable scientific version evaluation, rapid issue detection, and a fast closed‑loop verification process for game updates.

AB testingCanary DeploymentGame Platform

0 likes · 14 min read

Intelligent Gray Release Data Analysis System Practice for Game Center

Continuous Delivery 2.0

Jun 28, 2022 · Product Management

Why Repeating the Same A/B Test Multiple Times Is Wrong and How to Conduct Reliable Experiments

Repeating the same A/B test inflates false‑positive rates, creates inconsistent results, and hampers decision‑making, so a single well‑designed experiment with proper metrics, traffic allocation, analysis, and decision steps is essential for reliable product evaluation.

A/B TestingFalse Positiveexperiment design

0 likes · 6 min read

Why Repeating the Same A/B Test Multiple Times Is Wrong and How to Conduct Reliable Experiments

vivo Internet Technology

Jun 3, 2020 · Product Management

A Comprehensive Guide to AB Testing: Methodology and Implementation

This comprehensive guide explains AB testing fundamentals—from defining control and experimental groups and avoiding confounding factors, to calculating sample size, selecting ratio‑based metrics, tracking data, monitoring experiments, and analyzing statistical significance—providing a step‑by‑step methodology for data‑driven product optimization.

A/B Testing MethodologyAB testingExperimental design

0 likes · 14 min read

A Comprehensive Guide to AB Testing: Methodology and Implementation

Didi Tech

Aug 10, 2017 · Fundamentals

Understanding Hypothesis Testing and Statistical Significance in A/B Experiments

The article explains hypothesis testing in A/B experiments, describing null and alternative hypotheses, type I and II errors, p‑values, statistical significance versus practical impact, confidence intervals, statistical power, sample‑size planning, and a checklist for interpreting results responsibly.

A/B Testingconfidence intervalhypothesis testing

0 likes · 15 min read

Understanding Hypothesis Testing and Statistical Significance in A/B Experiments