experiment platform

DataFunSummit

Aug 18, 2024 · Artificial Intelligence

Challenges and Solutions in Recommendation AB Testing on Xiaohongshu's Experiment Platform

The article examines the key challenges of recommendation AB testing at Xiaohongshu—including change stability, single‑experiment precision, and multi‑strategy packaging—and presents a series of engineering and statistical solutions such as SDK‑based AB architecture, virtual PreAA experiments, CUPED/DID adjustments, and reverse experiments to improve reliability and metric impact.

AB testingCUPEDPreAA

0 likes · 15 min read

Challenges and Solutions in Recommendation AB Testing on Xiaohongshu's Experiment Platform

DataFunSummit

Dec 6, 2023 · Artificial Intelligence

Huya's Experiment Science Platform: Causal Inference, AB Testing, and Uplift Modeling Practices

Huya’s data‑driven experiment platform showcases how causal inference, AB testing, and uplift modeling are applied to advertising, user activation, and growth scenarios, detailing platform evolution, metric design, statistical challenges, and practical solutions such as multi‑test correction, CUPED, RTA, and propensity‑score methods.

AB testingcausal inferencedata science

0 likes · 18 min read

Huya's Experiment Science Platform: Causal Inference, AB Testing, and Uplift Modeling Practices

DataFunSummit

Jul 2, 2023 · Big Data

Building a One‑Stop AB Testing Platform at NetEase Cloud Music: Architecture, Metric Infrastructure, Scientific Evaluation, and Efficiency

The article describes how NetEase Cloud Music designed and deployed a comprehensive AB testing platform, covering system infrastructure, metric modeling, scientific experiment validation (including SRM mitigation and statistical power), and operational efficiency improvements to support rapid product iteration across multiple devices.

AB testingBig DataData Infrastructure

0 likes · 13 min read

Building a One‑Stop AB Testing Platform at NetEase Cloud Music: Architecture, Metric Infrastructure, Scientific Evaluation, and Efficiency

DataFunTalk

Apr 21, 2023 · Product Management

Best Practices for A/B Testing Platforms: Business Applicability, Internal Use Cases, Industry Examples, and Sustainable Experiment Culture

This article presents a comprehensive guide to A/B testing platforms, covering their business applicability, internal implementations at ByteDance, industry-specific case studies, platform architecture, experiment types, and strategies for building a sustainable experiment culture within organizations.

A/B testingbest practicescase studies

0 likes · 17 min read

Best Practices for A/B Testing Platforms: Business Applicability, Internal Use Cases, Industry Examples, and Sustainable Experiment Culture

DataFunSummit

Apr 12, 2023 · Product Management

Best Practices for A/B Testing Platforms: Business Applicability, Internal & Industry Cases, and Sustainable Experiment Culture

This article presents a comprehensive guide to A/B testing platforms, covering their business applicability, internal and external use cases across industries, detailed platform architecture, experiment types, statistical reporting, analysis tools, feature flag management, and recommendations for building a sustainable experiment culture within organizations.

A/B testingbest practicesdata-driven

0 likes · 19 min read

Best Practices for A/B Testing Platforms: Business Applicability, Internal & Industry Cases, and Sustainable Experiment Culture

DataFunTalk

Mar 26, 2023 · Fundamentals

Building and Applying an Experiment Platform: A/B Testing, System Integration, and Analysis

This article shares practical experiences and insights on building an experiment platform, covering A/B testing applications, platform selection, broader business use cases, rapid system integration, and robust experiment analysis techniques such as pre‑AA checks, SRM, Welch’s T and Z tests.

A/B testingMetric Managementdata analysis

0 likes · 19 min read

Building and Applying an Experiment Platform: A/B Testing, System Integration, and Analysis

DataFunTalk

Nov 27, 2022 · Product Management

Challenges of Traditional Experiment Systems and the Vision for Next‑Generation Evaluation Platforms

The article examines why classic A/B testing frameworks struggle with modern internet services—highlighting issues of intervention, measurement, and analysis—while proposing an observational, dynamic, and decision‑oriented next‑generation experiment system that leverages statistical learning and Bayesian optimization.

A/B testingBayesian Optimizationexperiment platform

0 likes · 11 min read

Challenges of Traditional Experiment Systems and the Vision for Next‑Generation Evaluation Platforms

DataFunTalk

Nov 25, 2022 · Operations

Overview of Volcano Engine A/B Experiment System Platform

This article presents a comprehensive overview of Volcano Engine's A/B testing platform, detailing its four core stages—reliable experiment system, efficient data construction, scientific statistical analysis, and fine-grained governance—while explaining execution components, data pipelines, statistical methods, and operational best practices for large‑scale experimentation.

A/B testingBig Datadata pipeline

0 likes · 16 min read

Overview of Volcano Engine A/B Experiment System Platform

vivo Internet Technology

Nov 16, 2022 · Big Data

Vivo Hawking A/B Experiment Platform: Architecture, Practices, and Solutions

The Vivo Hawking platform provides a company‑wide, one‑stop A/B testing solution with a layered architecture, covariate‑balanced split algorithms, real‑time monitoring, and unified SDKs for Android, Java and H5, enabling thousands of daily experiments, automated analysis, and rapid product iteration across multiple departments.

A/B testingCovariate balancingHive

0 likes · 22 min read

Vivo Hawking A/B Experiment Platform: Architecture, Practices, and Solutions

DataFunSummit

Oct 28, 2022 · Big Data

Design and Practice of a Risk Control Experiment Platform at Du Xiaoman

The article introduces the business background, architectural design, evolution challenges, and step‑by‑step methodology for building and operating a risk‑control experiment platform that supports online and offline A/B testing, data analysis, and strategy iteration in internet finance.

A/B testingdata analysisexperiment platform

0 likes · 12 min read

Design and Practice of a Risk Control Experiment Platform at Du Xiaoman

DataFunTalk

Oct 22, 2022 · Big Data

Design and Practice of a Risk Control Experiment Platform at Du Xiaoman

This article explains the background, architecture, challenges, and step‑by‑step design of a big‑data‑driven risk control experiment platform used for online and offline strategy testing in internet finance.

Big DataFinTechexperiment platform

0 likes · 12 min read

DevOps

Sep 16, 2022 · Operations

Rebuilding Uber’s Experimentation Platform: Architecture, Goals, and Lessons Learned

After more than a year of effort, Uber migrated its entire experimentation and feature‑flag ecosystem—including thousands of developers, dozens of partners, multiple mobile apps, and hundreds of services—to a new, unified platform that improves reliability, flexibility, and data quality while retiring over 50,000 legacy experiments.

A/B testingReliabilityUber

0 likes · 25 min read

Rebuilding Uber’s Experimentation Platform: Architecture, Goals, and Lessons Learned

Xingsheng Youxuan Technology Community

Jul 13, 2022 · Frontend Development

How Picasso Simplifies Frontend A/B Testing with Efficient Flow and Custom Rules

This article explains how the Picasso platform provides a standardized, high‑efficiency A/B testing pipeline for front‑end developers, covering traffic allocation algorithms, flow reuse, orthogonal and mutually exclusive experiments, multi‑scenario rules, custom metrics, reporting, and future enhancements.

A/B testingCustom Metricsexperiment platform

0 likes · 12 min read

How Picasso Simplifies Frontend A/B Testing with Efficient Flow and Custom Rules

HomeTech

Mar 24, 2022 · Fundamentals

A/B Testing Platform Overview and Statistical Evaluation Methods

This article introduces the A/B testing platform used at AutoHome, detailing its architecture, experiment flow, traffic allocation strategies, and statistical evaluation techniques such as hypothesis testing, confidence intervals, and significance testing, to guide data‑driven decision making for recommendation system improvements.

A/B testingRecommendation systemsdata-driven decisions

0 likes · 9 min read

A/B Testing Platform Overview and Statistical Evaluation Methods

iQIYI Technical Product Team

Aug 27, 2021 · Backend Development

iQIYI AB Testing Platform: Architecture, Workflow, and Statistical Practices

iQIYI’s AB testing platform integrates a layered traffic‑splitting architecture, real‑time SDK and API delivery, log‑replay data collection, and rigorous T‑test statistical analysis to enable fast, reliable product, algorithm, and operations experiments, exemplified by a UI redesign that boosted watch time by 17.85%.

AB testingTraffic Splittingexperiment platform

0 likes · 12 min read

iQIYI AB Testing Platform: Architecture, Workflow, and Statistical Practices

Bitu Technology

Mar 12, 2021 · Backend Development

Building Popper: Tubi’s Scalable Experiment Platform for Data‑Driven Decision Making

At Tubi, the Popper experiment engine—a Scala‑based, Akka‑powered backend service—combined with a self‑serve UI, automated analysis pipelines, and rigorous health checks, enables teams across ML, mobile, and OTT to run scalable A/B tests, rapidly iterate, and make data‑driven product decisions.

A/B testingAkkaML

0 likes · 14 min read

Building Popper: Tubi’s Scalable Experiment Platform for Data‑Driven Decision Making

DeWu Technology

Feb 26, 2021 · Backend Development

Design and Implementation of an AB Testing Platform with Traffic Allocation Algorithms

The paper presents an AB‑testing platform that structures experiments into scenes, buckets, layers and traffic, uses a salted‑hash based allocation and a two‑step “multi‑withdraw‑fill” algorithm to adjust percentages while preserving user‑experiment stability, and describes a lightweight, cache‑centric system architecture with staggered config reloads and safeguards against database spikes and zombie nodes.

AB testingAlgorithmexperiment platform

0 likes · 11 min read

Design and Implementation of an AB Testing Platform with Traffic Allocation Algorithms

Youzan Coder

Jul 15, 2020 · Big Data

Design and Implementation of Youzan ABTest System for Data‑Driven Growth

Youzan created an internal A/B testing platform—combining Java/Node SDKs, a real‑time data pipeline, and a metadata‑driven workflow—to enable data‑driven product iteration, granular traffic allocation, automated logging, statistical analysis, and scalable growth insights across its merchant services, while planning further automation and integration.

A/B testingBig Datadata-driven growth

0 likes · 19 min read

Design and Implementation of Youzan ABTest System for Data‑Driven Growth

DataFunTalk

May 31, 2020 · Big Data

Adaptive Grouping Method for Improving AB Test Allocation Uniformity in Didi's Experiment Platform

This article introduces Didi's adaptive grouping algorithm, which enhances the uniformity of user allocation in AB experiments by replacing traditional complete randomization with a single-pass method that balances observed metrics across groups, and demonstrates its effectiveness through large‑scale experimental results.

AB testingAlgorithmDidi

0 likes · 11 min read

Adaptive Grouping Method for Improving AB Test Allocation Uniformity in Didi's Experiment Platform

Beike Product & Technology

Jan 10, 2019 · Backend Development

Design and Implementation of the AB Experiment Platform at Beike Zhaofang

The article details the design principles, layered traffic allocation model, architecture, data processing pipeline, and operational experience of the AB experiment platform used at Beike Zhaofang, highlighting its web, API, and storage layers, gray‑release capabilities, current limitations, and future improvements.

AB testingbackend architecturedata pipeline

0 likes · 15 min read

Design and Implementation of the AB Experiment Platform at Beike Zhaofang