BestHub
Sign in
Tagged articles

A/B testing

264 articles · Page 2 of 3
DataFunSummit
DataFunSummit
Apr 12, 2023 · Product Management

Best Practices for A/B Testing Platforms: Business Applicability, Internal & Industry Cases, and Sustainable Experiment Culture

This article presents a comprehensive guide to A/B testing platforms, covering their business applicability, internal and external use cases across industries, detailed platform architecture, experiment types, statistical reporting, analysis tools, feature flag management, and recommendations for building a sustainable experiment culture within organizations.

A/B testingExperiment Platformbest practices
0 likes · 19 min read
Best Practices for A/B Testing Platforms: Business Applicability, Internal & Industry Cases, and Sustainable Experiment Culture
DataFunSummit
DataFunSummit
Apr 6, 2023 · Game Development

Experiment-Driven Advertising and User Operations in Game Growth: Causal Inference, Uplift Modeling, and Practical Pitfalls

This article presents a data‑science‑focused guide on using causal inference and uplift models to drive overseas ad targeting and user‑operation decisions in games, covering audience selection, privacy‑aware exposure correction, bid optimization, experiment design pitfalls, network effects, and practical recommendations.

A/B testingAdvertisingUplift Modeling
0 likes · 18 min read
Experiment-Driven Advertising and User Operations in Game Growth: Causal Inference, Uplift Modeling, and Practical Pitfalls
Tencent Advertising Technology
Tencent Advertising Technology
Mar 28, 2023 · Operations

Experimental Design for Two-Sided Markets in Advertising Scenarios

This article discusses experimental design challenges in two-sided markets, particularly in advertising scenarios, and presents various methods including four-table experiments, counterfactual interleaving, and contingency table joint sampling to address issues like network effects and competition between supply and demand sides.

A/B testingAdvertisingExperimental Design
0 likes · 14 min read
Experimental Design for Two-Sided Markets in Advertising Scenarios
DataFunTalk
DataFunTalk
Mar 27, 2023 · Product Management

Metric System Analysis and Growth Practice: Building Indicator Systems, Designing Optimization Strategies, and Real-World Case Studies

This presentation explains how to construct a metric system, identify bottlenecks, design targeted growth strategies, and apply A/B testing through detailed examples from Volcengine's DataFinder and DataTester platforms, culminating in practical case studies from Douyin Group and other enterprises.

A/B testingCase studyData Analysis
0 likes · 15 min read
Metric System Analysis and Growth Practice: Building Indicator Systems, Designing Optimization Strategies, and Real-World Case Studies
DaTaobao Tech
DaTaobao Tech
Mar 24, 2023 · Industry Insights

How Taobao’s “Home‑Page as Venue” Redefined Big‑Sale Performance

This article details the technical design, challenges, and results of Taobao’s “Home‑Page as Venue” initiative, explaining how a new A/B‑page architecture replaced crowded promotional cards with a dynamic cat‑head entry, improved information‑flow efficiency, reduced development cost, and delivered measurable gains in traffic, conversion, and revenue during major sales events.

A/B testingE‑commerceTaobao
0 likes · 23 min read
How Taobao’s “Home‑Page as Venue” Redefined Big‑Sale Performance
Python Programming Learning Circle
Python Programming Learning Circle
Mar 22, 2023 · Fundamentals

Comparing Distributions Between Groups: Visualization and Statistical Methods in Python

This article demonstrates how to compare the distribution of a variable across control and treatment groups using Python, covering data generation, visual techniques such as boxplots, histograms, KDE, CDF, QQ and ridgeline plots, and a suite of statistical tests including t‑test, SMD, Mann‑Whitney, permutation, chi‑square, Kolmogorov‑Smirnov and ANOVA for both two‑group and multi‑group scenarios.

A/B testingDistribution ComparisonPython
0 likes · 21 min read
Comparing Distributions Between Groups: Visualization and Statistical Methods in Python
ByteDance Data Platform
ByteDance Data Platform
Mar 22, 2023 · Backend Development

How DataTester’s Open Platform Empowers Customizable A/B Testing for Enterprises

DataTester’s Open Platform provides a configurable, extensible A/B testing solution that integrates via OpenAPI, SDKs, and a visual integration console, enabling enterprises to customize experiments, reduce development costs, and streamline deployment while supporting diverse business needs through modular components and a material ecosystem.

A/B testingDataTesterOpenAPI
0 likes · 12 min read
How DataTester’s Open Platform Empowers Customizable A/B Testing for Enterprises
Zhuanzhuan Tech
Zhuanzhuan Tech
Mar 8, 2023 · Product Management

A Comprehensive Guide to A/B Testing: Principles, Design, Metrics, and Decision Making

This article explains the fundamentals of A/B testing, why it is essential for data‑driven product decisions, how to design and run experiments—including hypothesis formulation, metric selection, sample size calculation, traffic segmentation, and duration planning—and how to analyze results using T‑tests, P‑values, and structured decision processes.

A/B testingdecision makingexperiment design
0 likes · 15 min read
A Comprehensive Guide to A/B Testing: Principles, Design, Metrics, and Decision Making
Architecture Digest
Architecture Digest
Mar 3, 2023 · Artificial Intelligence

Evolution and Architecture of Vivo's Game Recommendation System

This article chronicles the development, architectural challenges, and engineering solutions of a large‑scale game recommendation platform, covering background, initial models, business growth, caching strategies, GC optimization, rate‑limiting, fine‑grained operations, multi‑path recall, A/B testing, and future intelligent enhancements.

A/B testinggame recommendationsystem architecture
0 likes · 20 min read
Evolution and Architecture of Vivo's Game Recommendation System
Python Programming Learning Circle
Python Programming Learning Circle
Feb 24, 2023 · Fundamentals

Comparing Distributions Between Groups: Visualization and Statistical Methods in Python

This article demonstrates how to compare the distribution of a variable across treatment and control groups using Python, covering data simulation, visual techniques such as boxplots, histograms, KDE, CDF, QQ and ridgeline plots, and statistical tests including t‑test, SMD, Mann‑Whitney, permutation, chi‑square, KS and ANOVA.

A/B testingDistribution ComparisonPython
0 likes · 21 min read
Comparing Distributions Between Groups: Visualization and Statistical Methods in Python
DataFunSummit
DataFunSummit
Feb 23, 2023 · Artificial Intelligence

An Introduction to Causal Inference: Concepts, Methods, and Real‑World Applications

This article provides a comprehensive overview of causal inference, explaining its definition, the distinction between correlation and causation, classic pitfalls such as Simpson's paradox, key metrics like ATE and ATT, experimental designs, bias mitigation techniques, and practical case studies from content platforms and the Titanic dataset.

A/B testingBias Mitigationcausal inference
0 likes · 22 min read
An Introduction to Causal Inference: Concepts, Methods, and Real‑World Applications
vivo Internet Technology
vivo Internet Technology
Feb 22, 2023 · Backend Development

Game Recommendation System: Architecture, Models, Scaling, and Operational Practices

The article details the design, evolution, and operational practices of Vivo’s large‑scale game recommendation platform, covering its initial rule‑based model, layered strategy framework, multi‑level caching, GC tuning, rate‑limiting, fine‑grained A/B testing, multi‑path recall, dynamic exposure control, and future intelligent extensions.

A/B testingBackend ArchitectureGarbage Collection
0 likes · 17 min read
Game Recommendation System: Architecture, Models, Scaling, and Operational Practices
DataFunSummit
DataFunSummit
Feb 9, 2023 · Operations

Designing Experiments for Bilateral Markets in Advertising Platforms

This article explains how to design and evaluate experiments for bilateral markets in advertising platforms, covering the limitations of traditional randomization, the four‑cell traffic‑advertisement experiment, various mitigation strategies such as counterfactual interleaving and joint sampling, and the use of a simulation system to validate methods.

A/B testingadvertising experimentbilateral market
0 likes · 15 min read
Designing Experiments for Bilateral Markets in Advertising Platforms
DataFunTalk
DataFunTalk
Feb 6, 2023 · Product Management

A Comprehensive Guide to A/B Testing: Principles, Methods, and Applications

This article explains the scientific foundations, historical origins, statistical principles, implementation techniques, and practical applications of A/B testing as a data‑driven growth tool for product optimization, algorithm iteration, and marketing decisions in modern internet companies.

A/B testingdata-driven growthonline experiments
0 likes · 26 min read
A Comprehensive Guide to A/B Testing: Principles, Methods, and Applications
DataFunSummit
DataFunSummit
Feb 5, 2023 · Artificial Intelligence

Key Takeaways from the Causal Inference Summit: Motivation, Applications, Challenges, and Links to A/B Testing, Machine Learning, and Deep Learning

After attending the DataFun causal inference summit, this article outlines why causal analysis matters, its typical use cases, practical challenges, its relationship with A/B testing, and how it integrates with machine learning and deep learning to improve decision‑making and model robustness.

A/B testingUplift Modelingcausal inference
0 likes · 10 min read
Key Takeaways from the Causal Inference Summit: Motivation, Applications, Challenges, and Links to A/B Testing, Machine Learning, and Deep Learning
DataFunTalk
DataFunTalk
Jan 23, 2023 · Fundamentals

Understanding A/B Testing: Purpose, Process, and Practical Examples

A/B testing is a scientific method for product iteration that uses random user grouping, traffic segmentation, and metric analysis to derive representative conclusions, widely applied across major tech companies for evaluating ROI, with detailed workflow, example scenarios, and guidance on design and analysis.

A/B testingproduct analytics
0 likes · 5 min read
Understanding A/B Testing: Purpose, Process, and Practical Examples
DataFunTalk
DataFunTalk
Jan 5, 2023 · Big Data

Five Optimization Strategies for Improving DataTester Query Performance

This article describes how DataTester, Volcano Engine's A/B testing platform, achieved over four‑fold query speedup by applying five technical optimizations—including pre‑aggregation, join reduction, GroupBy redesign, AU‑metric caching, and asynchronous query handling—targeting both data construction and execution layers.

A/B testingClickHouseDataTester
0 likes · 12 min read
Five Optimization Strategies for Improving DataTester Query Performance
Dada Group Technology
Dada Group Technology
Dec 30, 2022 · Fundamentals

Ensuring Trustworthy A/B Experiments: Architecture, Balance Checks, Log Consistency, Automated Significance Testing, and Result Interpretation

This article discusses how to improve the reliability of online A/B experiments by designing robust architecture, evaluating group balance with orthogonal testing, ensuring consistent front‑end/back‑end logging, automating statistical significance checks, reducing group imbalance, and interpreting results using causal trees.

A/B testingData Collectioncausal trees
0 likes · 12 min read
Ensuring Trustworthy A/B Experiments: Architecture, Balance Checks, Log Consistency, Automated Significance Testing, and Result Interpretation
DataFunTalk
DataFunTalk
Dec 27, 2022 · Backend Development

Private Deployment Architecture, Challenges, and Solutions for Volcano Engine A/B Testing (DataTester)

This article details the private‑deployment architecture of Volcano Engine A/B Testing (DataTester), outlines three major challenges—version management, performance optimization, and stability—and explains the branch‑logic, release pipeline, model‑optimization, and pre‑aggregation solutions implemented to enable reliable, low‑resource SaaS‑like operation in on‑premise clusters.

A/B testingarchitecturebackend
0 likes · 12 min read
Private Deployment Architecture, Challenges, and Solutions for Volcano Engine A/B Testing (DataTester)
Past Memory Big Data
Past Memory Big Data
Dec 27, 2022 · Operations

How Volcano Engine DataTester Handles Private Deployment: Architecture, Challenges, and Business‑Driven Solutions

This article details Volcano Engine DataTester's private deployment architecture, the version‑management, performance, and stability challenges encountered, and the business‑oriented solutions—including branch strategies, pipeline automation, ClickHouse model optimizations, and multi‑level caching—that enable reliable, efficient A/B testing in on‑premise environments.

A/B testingAnsibleClickHouse
0 likes · 13 min read
How Volcano Engine DataTester Handles Private Deployment: Architecture, Challenges, and Business‑Driven Solutions
DataFunTalk
DataFunTalk
Dec 26, 2022 · Artificial Intelligence

A Review of Causal Inference Methods: Potential Outcomes, Structural Causal Models, and Recent Advances

This article reviews the two main streams of causal inference—potential‑outcome (Rubin) models and structural causal (Pearl) diagrams—covers classic techniques such as A/B testing, instrumental variables, matching, difference‑in‑differences, synthetic controls, matrix completion, heterogeneous treatment effect estimation, and discusses modern machine‑learning‑based approaches and causal discovery algorithms.

A/B testingcausal inferenceeconometrics
0 likes · 33 min read
A Review of Causal Inference Methods: Potential Outcomes, Structural Causal Models, and Recent Advances
Architecture & Thinking
Architecture & Thinking
Dec 26, 2022 · Cloud Native

Microservice Traffic Mastery: Canary, A/B Testing, and Service Mesh

This article explores essential microservice traffic management techniques—including canary releases, A/B testing, and traffic shading—detailing their value, implementation steps, and practical examples using Service Mesh and Istio, with code snippets and diagrams to illustrate routing based on request attributes.

A/B testingIstio
0 likes · 7 min read
Microservice Traffic Mastery: Canary, A/B Testing, and Service Mesh
58UXD
58UXD
Dec 22, 2022 · Product Management

What Drives Clicks? Insights from 58’s Feed Card Experiments

This article details a series of design experiments on 58’s homepage feed cards, analyzing how image realism, benefit labeling, and location presentation affect user click-through rates, revealing that authentic visuals, precise benefit information, and intuitive positioning outperform generic designs, and outlines a cyclical iteration process for continuous improvement.

A/B testingUI researchfeed design
0 likes · 12 min read
What Drives Clicks? Insights from 58’s Feed Card Experiments
DataFunTalk
DataFunTalk
Dec 8, 2022 · Product Management

Improving New User Retention in a Video App through A/B Testing: A Case Study

This article presents a detailed case study of how a video app team used two rounds of A/B testing with different swipe‑up guide designs to diagnose retention issues, refine the user onboarding experience, and ultimately achieve significant improvements in new‑user retention and engagement metrics.

A/B testingData AnalysisMobile App
0 likes · 10 min read
Improving New User Retention in a Video App through A/B Testing: A Case Study
DataFunTalk
DataFunTalk
Dec 6, 2022 · Databases

Performance Optimization of Apache Doris for A/B Experiment Queries at Xiaomi

This article analyzes the performance bottlenecks of A/B experiment report queries on Apache Doris at Xiaomi, presents data-driven insights on query latency, field usage, and experiment ID matching, and details a series of optimizations—including pre‑aggregation, materialized views, bitmap deduplication, and schema redesign—that reduced query times by up to 60× and lowered cluster load.

A/B testingApache DorisBitmap
0 likes · 17 min read
Performance Optimization of Apache Doris for A/B Experiment Queries at Xiaomi
58UXD
58UXD
Dec 5, 2022 · Product Management

When Data‑Driven Design Misleads: Insights from Google’s 41‑Shade Blue Test

This article examines how data‑driven design experiments—like Google’s 41‑shade blue link test and other color A/B studies—reveal both the power and pitfalls of relying solely on metrics, urging designers to balance data with intuition, broader context, and thoughtful KPI selection.

A/B testingKPI selectionUX Metrics
0 likes · 11 min read
When Data‑Driven Design Misleads: Insights from Google’s 41‑Shade Blue Test
58UXD
58UXD
Nov 30, 2022 · Operations

How Data‑Driven Design Boosts Click‑Through Rates in a Recruitment App

This article presents a data‑driven operational design case study for 58APP, detailing a five‑step testing framework, A/B experiments on UI elements such as real versus 3D personas and copy versus graphics, and the resulting insights that significantly improved click‑through and conversion rates.

A/B testingRecruitment Appclick-through rate
0 likes · 10 min read
How Data‑Driven Design Boosts Click‑Through Rates in a Recruitment App
DataFunTalk
DataFunTalk
Nov 30, 2022 · Big Data

Design and Practice of Yanxuan A/B Scientific Experiment Platform

The article presents the design, scientific methodology, system architecture, and case studies of Yanxuan's A/B testing platform, detailing how statistical principles, automated tracking, traffic allocation models, and unified reporting accelerate decision‑making and reduce development effort in e‑commerce experiments.

A/B testingAutomationData Pipeline
0 likes · 15 min read
Design and Practice of Yanxuan A/B Scientific Experiment Platform
DataFunTalk
DataFunTalk
Nov 27, 2022 · Product Management

Challenges of Traditional Experiment Systems and the Vision for Next‑Generation Evaluation Platforms

The article examines why classic A/B testing frameworks struggle with modern internet services—highlighting issues of intervention, measurement, and analysis—while proposing an observational, dynamic, and decision‑oriented next‑generation experiment system that leverages statistical learning and Bayesian optimization.

A/B testingExperiment Platformbayesian optimization
0 likes · 11 min read
Challenges of Traditional Experiment Systems and the Vision for Next‑Generation Evaluation Platforms
DataFunTalk
DataFunTalk
Nov 25, 2022 · Operations

Overview of Volcano Engine A/B Experiment System Platform

This article presents a comprehensive overview of Volcano Engine's A/B testing platform, detailing its four core stages—reliable experiment system, efficient data construction, scientific statistical analysis, and fine-grained governance—while explaining execution components, data pipelines, statistical methods, and operational best practices for large‑scale experimentation.

A/B testingBig DataData Pipeline
0 likes · 16 min read
Overview of Volcano Engine A/B Experiment System Platform
Bitu Technology
Bitu Technology
Nov 18, 2022 · Fundamentals

Tubi’s Switchback Experiment Platform: Design, Challenges, and Solutions

The article describes Tubi’s internal experimentation platform, explaining how traditional user‑group A/B tests can suffer from network interference and how Switchback experiments—time‑window based designs—address these issues, detailing their implementation, statistical methods, and the practical challenges overcome.

A/B testingSwitchback experimentscausal inference
0 likes · 12 min read
Tubi’s Switchback Experiment Platform: Design, Challenges, and Solutions
DataFunSummit
DataFunSummit
Oct 28, 2022 · Big Data

Design and Practice of a Risk Control Experiment Platform at Du Xiaoman

The article introduces the business background, architectural design, evolution challenges, and step‑by‑step methodology for building and operating a risk‑control experiment platform that supports online and offline A/B testing, data analysis, and strategy iteration in internet finance.

A/B testingData AnalysisExperiment Platform
0 likes · 12 min read
Design and Practice of a Risk Control Experiment Platform at Du Xiaoman
DevOps
DevOps
Sep 16, 2022 · Operations

Rebuilding Uber’s Experimentation Platform: Architecture, Goals, and Lessons Learned

After more than a year of effort, Uber migrated its entire experimentation and feature‑flag ecosystem—including thousands of developers, dozens of partners, multiple mobile apps, and hundreds of services—to a new, unified platform that improves reliability, flexibility, and data quality while retiring over 50,000 legacy experiments.

A/B testingExperiment PlatformUber
0 likes · 25 min read
Rebuilding Uber’s Experimentation Platform: Architecture, Goals, and Lessons Learned
ByteDance Data Platform
ByteDance Data Platform
Sep 7, 2022 · Product Management

How to Calculate Minimum Sample Size for Reliable A/B Tests

This article explains common pain points in A/B testing, introduces essential statistical concepts such as sampling distribution, parameter estimation, confidence intervals, and hypothesis testing, and provides step‑by‑step formulas and a concrete example for calculating the minimum sample size needed to run a trustworthy experiment.

A/B testinghypothesis testingproduct experimentation
0 likes · 14 min read
How to Calculate Minimum Sample Size for Reliable A/B Tests
DeWu Technology
DeWu Technology
Aug 29, 2022 · Fundamentals

Fundamentals of Statistics for A/B Testing and Its Application in the DeWu Platform

A solid grasp of basic statistical concepts—such as populations, samples, means, variance, probability distributions, the Central Limit Theorem, and hypothesis testing—enables designers of A/B experiments to correctly size samples, interpret p‑values and confidence intervals, and reliably deploy DeWu’s integrated platform for automated experiment allocation, metric monitoring, and one‑click reporting, ultimately driving data‑driven product decisions.

A/B testingconfidence intervaldata-driven
0 likes · 13 min read
Fundamentals of Statistics for A/B Testing and Its Application in the DeWu Platform
Huolala Tech
Huolala Tech
Aug 18, 2022 · R&D Management

How Huolala Built a Scalable A/B Testing Platform with Five Allocation Algorithms

Huolala’s A/B testing platform, serving over 200 business scenarios and thousands of experiments, combines a three‑stage workflow with a modular architecture—including a configuration console, SDK for traffic routing and data collection, and a robust data‑recovery pipeline—while offering five distinct allocation algorithms to ensure scientific experiment results.

A/B testingExperiment Platformalgorithm design
0 likes · 17 min read
How Huolala Built a Scalable A/B Testing Platform with Five Allocation Algorithms
DataFunTalk
DataFunTalk
Aug 18, 2022 · Fundamentals

Typical Applications and Classic Cases of A/B Testing

This article explains the origins of online A/B testing, outlines three typical product scenarios—recommendation, operations, and UI/UX—and presents classic case studies from companies like Bing, Google, Amazon, and TikTok to illustrate how controlled experiments drive data‑driven product optimization and measurable business impact.

A/B testingGrowth Hackingcase studies
0 likes · 15 min read
Typical Applications and Classic Cases of A/B Testing
58UXD
58UXD
Jul 29, 2022 · Product Management

How Systematic Thinking Boosts Conversion Rates in Family Service Apps

This article explains the concept of conversion rate, its significance for product success, and presents a systematic, full‑process approach to identify and optimize key decision points—covering user and scenario analysis, metric definition, business and user research, and concrete design strategies for entry, detail, and submission stages.

A/B testingbehavioral modelsconversion rate
0 likes · 17 min read
How Systematic Thinking Boosts Conversion Rates in Family Service Apps
Continuous Delivery 2.0
Continuous Delivery 2.0
Jul 28, 2022 · Product Management

How Netflix Uses A/B Testing to Drive Product Decisions

This article explains how Netflix applies large‑scale A/B experiments and scientific methods to make data‑driven product decisions, describing its decision‑making frameworks, the role of hypothesis testing, and the upcoming topics in the series on experimentation.

A/B testingNetflixdata-driven
0 likes · 5 min read
How Netflix Uses A/B Testing to Drive Product Decisions
Xingsheng Youxuan Technology Community
Xingsheng Youxuan Technology Community
Jul 13, 2022 · Frontend Development

How Picasso Simplifies Frontend A/B Testing with Efficient Flow and Custom Rules

This article explains how the Picasso platform provides a standardized, high‑efficiency A/B testing pipeline for front‑end developers, covering traffic allocation algorithms, flow reuse, orthogonal and mutually exclusive experiments, multi‑scenario rules, custom metrics, reporting, and future enhancements.

A/B testingExperiment PlatformFrontend
0 likes · 12 min read
How Picasso Simplifies Frontend A/B Testing with Efficient Flow and Custom Rules
Architecture Digest
Architecture Digest
Jul 12, 2022 · Big Data

Intelligent Gray Release Data System for Vivo Game Center: Methodology and Solutions

This article presents Vivo Game Center's end‑to‑end intelligent gray‑release data system, detailing its experimental mindset, statistical methods, data models, and product solutions that ensure scientific version evaluation, project progress, and rapid issue closure through root‑cause analysis and full‑process automation.

A/B testingData AnalysisRoot Cause Analysis
0 likes · 18 min read
Intelligent Gray Release Data System for Vivo Game Center: Methodology and Solutions
ITPUB
ITPUB
Jul 2, 2022 · Fundamentals

How Vivo Built an Intelligent Gray‑Release Data System for Faster, Scientific Game Updates

This article details Vivo Game Center's end‑to‑end intelligent gray‑release data framework—covering experiment design, statistical methods, data models, and automated product solutions—to ensure scientific version evaluation, accelerate project timelines, and quickly close the gray‑testing loop.

A/B testingData AnalyticsRoot Cause Analysis
0 likes · 16 min read
How Vivo Built an Intelligent Gray‑Release Data System for Faster, Scientific Game Updates
58UXD
58UXD
May 24, 2022 · Product Management

How to Turn Design Ideas into Data‑Driven Results: A Step‑by‑Step Guide

This article explains why designers must master data analysis, defines what “design data analysis” means, and walks through a three‑step framework—data splitting, tracking, and analysis—illustrated with practical e‑commerce and recruitment case studies to boost product metrics and retention.

A/B testingData AnalysisUser Retention
0 likes · 12 min read
How to Turn Design Ideas into Data‑Driven Results: A Step‑by‑Step Guide
DaTaobao Tech
DaTaobao Tech
May 17, 2022 · Artificial Intelligence

Self-Supervised Learning for Image Embeddings in Recommendation Systems: SwAV and M6 Applications at Meiping Meiwu

The paper demonstrates how self‑supervised models SwAV and M6 generate high‑quality image and multimodal embeddings for Meiping Meiwu’s recommendation system, delivering notable gains in scene/style consistency, ranking AUC, classification and retrieval performance, especially for cold‑start items, and achieving measurable production lifts.

A/B testingM6 multimodalRecommendation Systems
0 likes · 15 min read
Self-Supervised Learning for Image Embeddings in Recommendation Systems: SwAV and M6 Applications at Meiping Meiwu
DataFunTalk
DataFunTalk
May 10, 2022 · Artificial Intelligence

Experimental Science and Causal Inference Forum – Sessions Overview at DataFun Summit 2022

The DataFun Summit 2022 features an Experimental Science and Causal Inference forum where leading data scientists from Didi, Tencent, Google, ByteDance, and others present deep technical talks on causal inference methods, A/B testing, game operations, and advertising experiments, offering practical insights and audience takeaways.

A/B testingAdvertisingdata science
0 likes · 10 min read
Experimental Science and Causal Inference Forum – Sessions Overview at DataFun Summit 2022
DaTaobao Tech
DaTaobao Tech
Apr 26, 2022 · Artificial Intelligence

Optimization of Recall, Ranking, and Downward Modeling for the "Every Square Every House" Infinite-Scroll Light App

This article details a year‑long series of experiments on the Taobao “Every Square Every House” infinite‑scroll light app, describing how added recall paths, a coarse‑ranking filter, multi‑task MMOE sorting, a lightweight down‑scroll predictor, and relevance‑enhanced features together boosted click‑through, scroll depth and per‑user engagement by double‑digit percentages.

A/B testingModel Optimizationinfinite-scroll
0 likes · 14 min read
Optimization of Recall, Ranking, and Downward Modeling for the "Every Square Every House" Infinite-Scroll Light App
58UXD
58UXD
Apr 19, 2022 · Product Management

Boost Holiday Service Orders: A Multi‑Service Package Design Case Study

This article analyzes how a home‑cleaning platform increased Chinese New Year service orders by designing a multi‑service ordering flow, comparing four packaging options, selecting a package‑recommendation model, eliminating three key friction points, and validating the solution through A/B testing and metric improvements.

A/B testingUX optimizatione-commerce
0 likes · 10 min read
Boost Holiday Service Orders: A Multi‑Service Package Design Case Study
HomeTech
HomeTech
Mar 24, 2022 · Fundamentals

A/B Testing Platform Overview and Statistical Evaluation Methods

This article introduces the A/B testing platform used at AutoHome, detailing its architecture, experiment flow, traffic allocation strategies, and statistical evaluation techniques such as hypothesis testing, confidence intervals, and significance testing, to guide data‑driven decision making for recommendation system improvements.

A/B testingExperiment Platformdata-driven decisions
0 likes · 9 min read
A/B Testing Platform Overview and Statistical Evaluation Methods
Kuaishou Tech
Kuaishou Tech
Mar 16, 2022 · Artificial Intelligence

Multi-Dimensional Causal Forest Model for Heterogeneous Treatment Effects in Marketing

This paper introduces a novel multi-dimensional causal forest model combined with efficient integer programming algorithms to estimate heterogeneous treatment effects (HTE) in marketing scenarios, outperforming traditional tree-based methods through improved handling of intervention heterogeneity and resource allocation optimization.

A/B testingMarketing AlgorithmsTencent Research
0 likes · 7 min read
Multi-Dimensional Causal Forest Model for Heterogeneous Treatment Effects in Marketing
DevOps
DevOps
Feb 24, 2022 · Product Management

A/B Testing: Motivation, Architecture, Best Practices, and Future Outlook

This article explains why A/B testing is essential for data‑driven decision making, describes the Volcano Engine A/B testing system architecture, outlines practical experiment design, statistical analysis methods, real‑world case studies, and forecasts industry and technical trends for the practice.

A/B testingData-driven Decisionexperiment design
0 likes · 15 min read
A/B Testing: Motivation, Architecture, Best Practices, and Future Outlook
DaTaobao Tech
DaTaobao Tech
Feb 9, 2022 · User Experience Design

How Taobao Redefined Shopping for Seniors: The ‘Elder Mode’ Design Journey

Taobao’s senior‑friendly “Elder Mode” redesign tackles the challenges of an aging population by enlarging fonts, simplifying information, adding voice search, creating a walking‑based game, and implementing seamless version switching, all backed by extensive user research, technical solutions, and A/B experiments to improve the digital shopping experience for older users.

A/B testingMobile UIproduct customization
0 likes · 21 min read
How Taobao Redefined Shopping for Seniors: The ‘Elder Mode’ Design Journey
DataFunTalk
DataFunTalk
Jan 17, 2022 · Artificial Intelligence

Causal Analysis in Real Estate: Challenges, Methodology, and Practice at Beike

This article explains the wide applicability of causal inference, outlines three major challenges—correlation vs. causation, confounding factors, and selection bias—and demonstrates a scientific three‑level approach using examples such as smoking and a real‑world deployment of an intelligent client‑management tool at Beike, including experimental designs, results, and lessons learned.

A/B testingArtificial IntelligenceData Analysis
0 likes · 14 min read
Causal Analysis in Real Estate: Challenges, Methodology, and Practice at Beike
ByteDance Data Platform
ByteDance Data Platform
Jan 14, 2022 · Product Management

Why A/B Testing Matters: Theory, ByteDance Architecture & Best Practices

This article explains why A/B testing is crucial for data‑driven product decisions, outlines ByteDance’s A/B testing system architecture across multiple layers, describes client‑ and server‑side experiment workflows, shares statistical best practices, and presents real‑world case studies illustrating hypothesis generation, evaluation, and future industry trends.

A/B testingByteDancedata-driven
0 likes · 15 min read
Why A/B Testing Matters: Theory, ByteDance Architecture & Best Practices
Top Architect
Top Architect
Jan 3, 2022 · Operations

Gray Release (Canary Deployment) Design and Implementation Guide

This article explains the concept of gray release, outlines a simple architecture with essential components, describes common traffic-splitting strategies, shows how to implement control in Nginx and service layers, and discusses complex scenarios such as multi‑service and data‑centric deployments.

A/B testingBackend ArchitectureCanary Deployment
0 likes · 7 min read
Gray Release (Canary Deployment) Design and Implementation Guide
ByteDance Terminal Technology
ByteDance Terminal Technology
Nov 9, 2021 · Artificial Intelligence

Edge AI Video Preloading: Case Study and Implementation with ByteDance's Client AI Platform

This article presents a comprehensive case study of applying edge AI to video preloading on the Xigua Video platform, detailing scenario analysis, predictive modeling of user behavior, feature engineering, on‑device model inference, dynamic algorithm package deployment, experimental evaluation, and the resulting performance and cost improvements.

A/B testingModel Optimizationclient inference
0 likes · 18 min read
Edge AI Video Preloading: Case Study and Implementation with ByteDance's Client AI Platform
DataFunSummit
DataFunSummit
Nov 5, 2021 · Artificial Intelligence

Practical Insights into Online Experiment Design and Analysis at Tencent Lookpoint

The presentation offers a comprehensive overview of online experiment fundamentals, design variations, and real-world case studies from Tencent Lookpoint, emphasizing hypothesis validation, causal analysis, best practices, and actionable recommendations for improving product growth and decision‑making.

A/B testingRecommendation Systemscausal inference
0 likes · 20 min read
Practical Insights into Online Experiment Design and Analysis at Tencent Lookpoint
DataFunTalk
DataFunTalk
Nov 1, 2021 · Product Management

Online Experiment Design and Analysis: Practices, Case Studies, and Guidelines from Tencent Data Platform

This article presents a comprehensive overview of online experiment design and analysis, covering basic definitions, AB testing principles, complex experiment types, real-world case studies from Tencent's information flow platform, and practical guidelines for reliable experiment evaluation and product decision‑making.

A/B testingRecommendation Systemscausal inference
0 likes · 21 min read
Online Experiment Design and Analysis: Practices, Case Studies, and Guidelines from Tencent Data Platform
DevOps Cloud Academy
DevOps Cloud Academy
Oct 26, 2021 · Cloud Native

Progressive Delivery with Spinnaker: Principles, Architecture, and Implementation

This article explains the concept of progressive delivery, its industrial origins, how cloud‑native tools like Kubernetes, Traefik and Spinnaker enable automated A/B testing, canary and gray releases, and the practical benefits for developers, operations, and product teams when releasing features to millions of users.

A/B testingCI/CDCanary Release
0 likes · 9 min read
Progressive Delivery with Spinnaker: Principles, Architecture, and Implementation
JD.com Experience Design Center
JD.com Experience Design Center
Sep 28, 2021 · Product Management

How A/B Testing Boosted Click‑Through Rates Across Five Design Elements

This report details a series of over twenty A/B tests conducted across multiple product divisions, revealing how targeted design tweaks—such as contrasting guide buttons, fresh promotional visuals, prominent coupon styling, focused product ads, and interactive floor modules—significantly improve user click‑through rates and conversion.

A/B testingConversion OptimizationUX design
0 likes · 7 min read
How A/B Testing Boosted Click‑Through Rates Across Five Design Elements
转转QA
转转QA
Sep 26, 2021 · Big Data

A/B Testing Process Improvement and Validation Guide

This article outlines a comprehensive A/B testing workflow, covering historical issues, business test process improvements, detailed implementation steps, SQL validation scripts, data verification in analytics platforms, and practical notes to ensure accurate experiment data collection and analysis.

A/B testingBig DataSQL
0 likes · 10 min read
A/B Testing Process Improvement and Validation Guide
ByteDance SE Lab
ByteDance SE Lab
Sep 17, 2021 · Product Management

Why A/B Testing Matters: Cases, Architecture & Best Practices

This article explains why A/B testing is essential, illustrates real-world examples from ByteDance, details the multi-layer architecture of the Volcano Engine A/B testing system, outlines experiment design, implementation, statistical analysis, best practices, and future trends, providing a comprehensive guide for product teams.

A/B testingData Analysisexperiment design
0 likes · 18 min read
Why A/B Testing Matters: Cases, Architecture & Best Practices
Java Interview Crash Guide
Java Interview Crash Guide
Sep 16, 2021 · Artificial Intelligence

Inside Toutiao’s Recommendation Engine: Architecture, Features, and Safety

This article explains the architecture and key components of Toutiao’s recommendation system, covering system overview, content analysis, user tagging, evaluation methods, and content safety measures, and discusses practical implementation details such as feature engineering, model training, recall strategies, and online experimentation.

A/B testingcontent moderationfeature engineering
0 likes · 20 min read
Inside Toutiao’s Recommendation Engine: Architecture, Features, and Safety
Baidu Geek Talk
Baidu Geek Talk
Sep 8, 2021 · Industry Insights

How to Scale A/B Testing Platforms for Massive Teams and Data Volumes

This article examines the core challenges of running large‑scale A/B testing platforms—supporting thousands of engineers, generating fast reports from massive data sets, and reducing sampling variance—to enable data‑driven product decisions in the AI era.

A/B testingExperiment Platformdata-driven
0 likes · 2 min read
How to Scale A/B Testing Platforms for Massive Teams and Data Volumes
DataFunSummit
DataFunSummit
Sep 5, 2021 · Artificial Intelligence

Causal Inference and Experiment Design in Kuaishou Live Streaming: Methods and Case Studies

This article explains how Kuaishou applies causal inference frameworks, such as Rubin's potential outcomes and Pearl's causal graphs, together with machine‑learning techniques like double‑machine learning, causal forests, and meta‑learners to evaluate product features, recommendation strategies, and user behavior under complex network effects in live streaming.

A/B testingKuaishoucausal inference
0 likes · 14 min read
Causal Inference and Experiment Design in Kuaishou Live Streaming: Methods and Case Studies
Baidu MEUX
Baidu MEUX
Sep 3, 2021 · Product Management

Turning Search into Medical Beauty Knowledge: Baidu’s Lemon Aesthetic Case Study

This case study explores how Baidu’s Lemon Aesthetic brand leveraged Baidu App’s natural traffic to transform search queries into a structured medical‑beauty knowledge experience, using user research, KPI‑driven design, MVP validation, and A/B testing to boost engagement, conversion, and brand authority.

A/B testingMVPMedical Aesthetics
0 likes · 10 min read
Turning Search into Medical Beauty Knowledge: Baidu’s Lemon Aesthetic Case Study
Volcano Engine Developer Services
Volcano Engine Developer Services
Aug 19, 2021 · Product Management

Mastering A/B Testing: Architecture, Best Practices, and Real-World Insights

This article explains why A/B testing is essential, defines the methodology, details Volcano Engine's multi‑layer A/B testing architecture, outlines client and server experiment flows, shares statistical analysis practices, best‑practice guidelines, future trends, and answers common questions.

A/B testingdata-drivenexperiment design
0 likes · 17 min read
Mastering A/B Testing: Architecture, Best Practices, and Real-World Insights
DataFunTalk
DataFunTalk
Aug 12, 2021 · Artificial Intelligence

Causal Inference and Experiment Design in Kuaishou Live Streaming

This article presents Dr. Jin Yaran’s comprehensive overview of causal inference challenges, frameworks, and practical case studies—including DID, double machine learning, causal forests, and meta‑learners—applied to Kuaishou’s live‑streaming product, and discusses complex experimental designs such as bilateral and time‑slice experiments.

A/B testingData AnalysisKuaishou
0 likes · 15 min read
Causal Inference and Experiment Design in Kuaishou Live Streaming
TAL Education Technology
TAL Education Technology
Aug 12, 2021 · Fundamentals

Statistical Foundations and Practical Applications of A/B Testing

This article explains the statistical principles behind A/B testing, covering concepts such as populations, samples, parameters, hypothesis testing, significance levels, t‑tests, metric types, p‑value calculations, and real‑world examples to guide data‑driven product decisions.

A/B testingMetric Evaluationexperiment design
0 likes · 12 min read
Statistical Foundations and Practical Applications of A/B Testing
DataFunTalk
DataFunTalk
Jul 17, 2021 · Artificial Intelligence

Multi-Objective Modeling for CRM Opportunity Smart Allocation: Iterative Deep Learning Solutions

This article describes the evolution of a multi‑objective deep‑learning framework for automatically assigning CRM opportunities to salespeople, detailing five model versions—from an XGBoost baseline with sample weighting to advanced PLE‑based architectures—while reporting offline and online performance gains in both call‑out and connection‑out conversion rates.

A/B testingCRMRecommendation Systems
0 likes · 33 min read
Multi-Objective Modeling for CRM Opportunity Smart Allocation: Iterative Deep Learning Solutions
Alimama Tech
Alimama Tech
Jul 14, 2021 · Big Data

A/B Testing Framework for Online Experiments: Design, Implementation, Analysis, and Decision Making

The paper presents a comprehensive A/B testing framework for online experiments that guides practitioners through four stages—designing objectives and metrics, implementing random traffic allocation with robustness checks, evaluating effects using descriptive statistics and hypothesis testing, and making rollout decisions based on multidimensional significance and attribution analyses.

A/B testingData AnalysisExperimental Design
0 likes · 22 min read
A/B Testing Framework for Online Experiments: Design, Implementation, Analysis, and Decision Making
Top Architect
Top Architect
Jul 4, 2021 · Operations

Design and Implementation of a Simple Gray Release System

The article explains the concept of gray release, outlines a basic architecture with strategy configuration, execution, and service registry components, describes common traffic-splitting strategies, and details practical implementations using Nginx, gateway services, and complex scenarios involving data synchronization and message queues.

A/B testingOperationsbackend
0 likes · 7 min read
Design and Implementation of a Simple Gray Release System
TAL Education Technology
TAL Education Technology
Jul 1, 2021 · Big Data

Optimization of A/B Test Metric Computation Using Spark and ClickHouse

This article details the design and multi‑stage optimization of an A/B testing metric system, describing its product architecture, Spark‑based computation engine, ClickHouse OLAP layer, cumulative calculation improvements, and batch processing techniques that reduced processing time from hours to a few minutes for hundreds of experiments and metrics.

A/B testingBig DataClickHouse
0 likes · 8 min read
Optimization of A/B Test Metric Computation Using Spark and ClickHouse
DataFunTalk
DataFunTalk
Jun 28, 2021 · Fundamentals

Bayesian A/B Testing with PyMC3: A Practical Guide

This article introduces the motivation and logic behind A/B testing, highlights common misunderstandings of p‑values, and demonstrates how Bayesian A/B testing using PyMC3 can provide intuitive probability statements about which variant performs better, complete with Python code examples.

A/B testingBayesian statisticsPyMC3
0 likes · 12 min read
Bayesian A/B Testing with PyMC3: A Practical Guide
流利说 Design Team
流利说 Design Team
May 26, 2021 · Product Management

Turning Simple Check‑In Features into Powerful Growth Engines at Liulishuo

This case study reveals how Liulishuo’s design and product teams leveraged check‑in gamification, friend‑invitation flows, and strategic banner placements—backed by rigorous A/B testing and data‑driven insights—to boost user engagement, optimize information‑flow ads, and improve overall product performance.

A/B testingGrowth Hackinginformation flow ads
0 likes · 8 min read
Turning Simple Check‑In Features into Powerful Growth Engines at Liulishuo
21CTO
21CTO
Apr 20, 2021 · Fundamentals

Why A/B Testing Isn’t a Silver Bullet: Lessons from ByteDance’s Data‑Driven Journey

In a ByteDance tech open‑day talk, Yang Zhenyuan explains how clear, measurable goals, the limits of user‑profile and usage‑time metrics, and careful A/B test design are essential for data‑driven product decisions, while highlighting practical pitfalls and real‑world examples.

A/B testingProduct Developmentdata-driven
0 likes · 18 min read
Why A/B Testing Isn’t a Silver Bullet: Lessons from ByteDance’s Data‑Driven Journey
IT Architects Alliance
IT Architects Alliance
Apr 15, 2021 · Operations

Design and Implementation of a Simple Gray Release System

This article explains the concept of gray (canary) release, outlines a basic architecture with essential components, describes common gray release strategies such as header, cookie, and parameter based routing, and provides practical guidance for implementing gray releases using Nginx, gateway services, and handling complex scenarios like multi‑service and database migrations.

A/B testingCanary DeploymentMicroservices
0 likes · 7 min read
Design and Implementation of a Simple Gray Release System
Architect
Architect
Apr 11, 2021 · Operations

Gray Release (Canary Deployment): Definition, Design, Strategies, and Complex Scenarios

This article explains gray release (canary deployment) as an A/B testing‑style rollout, outlines a simple architecture with strategy configuration, execution, and service registry components, describes common traffic‑splitting strategies, and discusses implementation details for Nginx, gateway layers, and complex multi‑service and data‑centric scenarios.

A/B testingCanary DeploymentDeployment Strategy
0 likes · 6 min read
Gray Release (Canary Deployment): Definition, Design, Strategies, and Complex Scenarios
Liangxu Linux
Liangxu Linux
Mar 31, 2021 · Cloud Native

Mastering Canary Deployments with Ingress‑Nginx Annotations in Kubernetes

This guide explains how to use Ingress‑Nginx 0.21+ Canary annotations to perform blue‑green, canary, and A/B testing deployments on Kubernetes, covering annotation syntax, priority rules, and step‑by‑step examples with weight‑based, header‑based, and cookie‑based traffic splitting.

A/B testingBlue-GreenCanary Deployment
0 likes · 17 min read
Mastering Canary Deployments with Ingress‑Nginx Annotations in Kubernetes
DataFunTalk
DataFunTalk
Mar 18, 2021 · Fundamentals

Building Popper: Tubi’s Scalable Experimentation Platform

Tubi’s Popper platform combines a Scala‑based experiment engine, reproducible JSON‑stored configurations, a React UI, and data pipelines using Spark and Akka to enable fast, cross‑team A/B testing, automated analysis, health checks, and data‑driven decision making across mobile and OTT services.

A/B testingAkkaData pipelines
0 likes · 15 min read
Building Popper: Tubi’s Scalable Experimentation Platform
Bitu Technology
Bitu Technology
Mar 12, 2021 · Backend Development

Building Popper: Tubi’s Scalable Experiment Platform for Data‑Driven Decision Making

At Tubi, the Popper experiment engine—a Scala‑based, Akka‑powered backend service—combined with a self‑serve UI, automated analysis pipelines, and rigorous health checks, enables teams across ML, mobile, and OTT to run scalable A/B tests, rapidly iterate, and make data‑driven product decisions.

A/B testingAkkaExperiment Platform
0 likes · 14 min read
Building Popper: Tubi’s Scalable Experiment Platform for Data‑Driven Decision Making
MaGe Linux Operations
MaGe Linux Operations
Mar 7, 2021 · Cloud Native

Mastering Canary Deployments with Ingress-Nginx Annotations in Kubernetes

This article explains how to implement blue‑green and canary releases on Kubernetes using Ingress‑Nginx’s annotation‑based canary feature, covering weight‑based traffic splitting, header‑ and cookie‑driven routing, with step‑by‑step YAML examples and command‑line testing for validation.

A/B testingBlue-GreenCanary
0 likes · 17 min read
Mastering Canary Deployments with Ingress-Nginx Annotations in Kubernetes
Programmer DD
Programmer DD
Feb 18, 2021 · Operations

How Gray Release Enables Safe, Rapid Feature Rollouts in Production

This article explains the concept of gray release, outlines a simple architecture with essential components, describes common routing strategies, and shows how to implement gray releases using Nginx, gateway services, and complex multi‑service scenarios to ensure controlled, low‑risk deployments.

A/B testingDeployment StrategyOperations
0 likes · 7 min read
How Gray Release Enables Safe, Rapid Feature Rollouts in Production
Alibaba Terminal Technology
Alibaba Terminal Technology
Feb 5, 2021 · Frontend Development

How Intelligent UI Boosted Alibaba’s Holiday Sales by 10%+ Through User Preference Modeling

This article explains how Alibaba’s CBU team tackled decision overload by building an intelligent UI that uses user‑behavior and product‑preference models, replaces algorithmic cold‑start, reduces reliance on traffic, and delivers over 10% PVctr growth across multiple holiday campaigns through systematic tagging, low‑code material development, and rigorous A/B experimentation.

A/B testingAlgorithmic RecommendationUser Preference Modeling
0 likes · 20 min read
How Intelligent UI Boosted Alibaba’s Holiday Sales by 10%+ Through User Preference Modeling
58UXD
58UXD
Jan 7, 2021 · Product Management

How Redesigning a Commercial Purchase Page Boosted Conversion and Repeat Purchases

This case study details how a thorough redesign of 58's commercial purchase page—grounded in user interviews, behavior‑model alignment, clearer discount presentation, real reviews, and A/B testing—significantly increased purchase conversion rates, revenue, and customer satisfaction while informing future product improvements.

A/B testingConversion OptimizationUX design
0 likes · 9 min read
How Redesigning a Commercial Purchase Page Boosted Conversion and Repeat Purchases
Architecture Digest
Architecture Digest
Jan 4, 2021 · Operations

Design and Implementation of a Gray Release System

This article explains the concept of gray release, outlines a simple architecture with essential components, describes common strategies such as header, cookie, and parameter based routing, and provides detailed implementation guidance for Nginx, gateway, and complex multi‑service scenarios.

A/B testingOperationsgray release
0 likes · 7 min read
Design and Implementation of a Gray Release System
Top Architect
Top Architect
Dec 29, 2020 · Backend Development

Migrating OkCupid from REST to GraphQL: Process, Lessons, and Outcomes

The article details OkCupid’s year‑and‑a‑half migration from a REST API to a production GraphQL API, describing the four‑step process—page selection, schema construction, shadow requests, and A/B testing—along with lessons learned about error handling, business‑logic placement, and performance monitoring.

A/B testingAPI migrationApollo Server
0 likes · 9 min read
Migrating OkCupid from REST to GraphQL: Process, Lessons, and Outcomes
58UXD
58UXD
Dec 28, 2020 · Product Management

How an Online Appointment Redesign Boosted Conversion for a Local Services Platform

This case study examines how a Chinese local services platform tackled information asymmetry between providers and users by replacing phone calls with an online appointment system, introducing a prepaid reservation fee, redesigning UI to reduce friction, and using A/B testing to achieve significant conversion improvements.

A/B testingConversion Optimizationonline appointment
0 likes · 8 min read
How an Online Appointment Redesign Boosted Conversion for a Local Services Platform