BestHub
Sign in
Tagged articles
256 articles
Page 1 of 3
Woodpecker Software Testing
Woodpecker Software Testing
May 7, 2026 · Artificial Intelligence

How Prompt Testing Opens a New Dimension of AI Application Performance

The article explains why prompts, now treated as a measurable software interface, become a performance bottleneck in AI-native apps, and presents a four‑quadrant methodology—including observability, quantification, attribution, and governance—plus five concrete optimization tactics backed by real‑world case studies.

A/B testingLLM PerformancePrompt Engineering
0 likes · 8 min read
How Prompt Testing Opens a New Dimension of AI Application Performance
Woodpecker Software Testing
Woodpecker Software Testing
Apr 9, 2026 · Product Management

5 Hidden Pitfalls of A/B Test Automation in 2026

In 2026, AI‑driven A/B testing platforms became standard, cutting experiment cycles by 63% but raising false‑positive rates to 19.4%, and this article reveals five critical mistakes—from mistaking auto‑traffic split for true randomization to ignoring metric drift and business impact—that can undermine results.

A/B testingautomationexperiment design
0 likes · 8 min read
5 Hidden Pitfalls of A/B Test Automation in 2026
Woodpecker Software Testing
Woodpecker Software Testing
Apr 4, 2026 · Product Management

2026 A/B Testing Automation: Emerging Trends and Real‑World Practices

The article examines how 2026’s new A/B testing automation paradigm—combining dynamic traffic allocation, real‑time causal modeling, metric‑autonomy systems, and built‑in privacy compliance—dramatically shortens experiment cycles, boosts statistical power, and transforms experimentation from a manual chore into a scalable, trustworthy decision engine.

A/B testingGrowth Engineeringautomation
0 likes · 8 min read
2026 A/B Testing Automation: Emerging Trends and Real‑World Practices
Woodpecker Software Testing
Woodpecker Software Testing
Apr 3, 2026 · Product Management

From Experience‑Driven to Data‑Loop: How One SaaS Team Automated A/B Testing

The article details a mid‑size SaaS growth team’s transformation from manual, experience‑driven A/B testing to a fully automated, auditable end‑to‑end decision flow, describing the pitfalls of pseudo‑automation, a three‑layer automation engine, and cultural shifts that boosted experiment adoption from 41% to 89% and cut decision latency from 3.7 days to 4.2 hours.

A/B testingData-drivenExperiment-as-Code
0 likes · 8 min read
From Experience‑Driven to Data‑Loop: How One SaaS Team Automated A/B Testing
Geek Labs
Geek Labs
Apr 3, 2026 · Product Management

Essential AI Skills Every Product Manager Needs: From Ideation to Data Tracking

This guide lists six open‑source AI skills that help product managers turn vague ideas into concrete designs, write structured PRDs, break requirements into executable plans, set up rigorous A/B experiments, implement analytics tracking, and optimize new‑user onboarding, each with install commands and usage examples.

A/B testingAIAnalytics
0 likes · 9 min read
Essential AI Skills Every Product Manager Needs: From Ideation to Data Tracking
Shuge Unlimited
Shuge Unlimited
Mar 6, 2026 · Artificial Intelligence

Skill-Creator Update: 83.3% Trigger Success and 5 New Engineering Features

Anthropic's March 2026 skill‑creator update adds five engineering‑focused functions—Evals, Benchmark, multi‑agent parallelism, A/B testing, and trigger optimization—enabling systematic testing, performance tracking, and a reported 83.3% improvement in trigger success across public skills.

A/B testingAI agentsClaude
0 likes · 17 min read
Skill-Creator Update: 83.3% Trigger Success and 5 New Engineering Features
Woodpecker Software Testing
Woodpecker Software Testing
Mar 1, 2026 · Artificial Intelligence

Four Hidden Model Evaluation Pitfalls That Undermine AI Deployments

The article examines four common yet hidden model evaluation mistakes—confusing attractive metrics with business impact, using static test sets, ignoring statistical significance, and lacking fine‑grained attribution—illustrating each with real‑world cases and offering concrete practices to build a more robust, business‑aligned evaluation pipeline.

A/B testingAI deploymentMetrics
0 likes · 8 min read
Four Hidden Model Evaluation Pitfalls That Undermine AI Deployments
Alibaba Cloud Developer
Alibaba Cloud Developer
Feb 2, 2026 · Artificial Intelligence

Boosting A/B Experiment Automation: Prompt Engineering Achieves 80% Accuracy

This article details how a production‑grade prompt system powered by large language models was designed to replace manual A/B experiment inspection, introducing a six‑level priority decision tree, robust data preprocessing, and systematic bad‑case analysis that lifted automation accuracy from 68% to over 80% while providing clear, explainable recommendations.

A/B testingLLMPrompt Engineering
0 likes · 46 min read
Boosting A/B Experiment Automation: Prompt Engineering Achieves 80% Accuracy
Java Baker
Java Baker
Jan 31, 2026 · Backend Development

Mastering Gray Releases and A/B Testing: Strategies, Code, and Analytics

This article provides a comprehensive guide to gray releases and A/B testing, covering common scenarios, implementation methods, layered experiment design, hash-based bucket allocation, data collection workflows, statistical analysis, and practical Java and SQL code examples for reliable feature validation.

A/B testingbackend-developmentexperiment design
0 likes · 11 min read
Mastering Gray Releases and A/B Testing: Strategies, Code, and Analytics
Tech Freedom Circle
Tech Freedom Circle
Jan 22, 2026 · Operations

Designing Gray Release and A/B Testing for Safe Deployments and Winning Experiments

This article explains the fundamental differences between gray release and A/B testing, provides step‑by‑step guidance for implementing both strategies with Spring Cloud Gateway, Nacos and Kubernetes, and compares container‑level canary deployments with gateway‑level traffic routing to help you choose the right approach for reliable production releases.

A/B testingDeploymentKubernetes
0 likes · 43 min read
Designing Gray Release and A/B Testing for Safe Deployments and Winning Experiments
We-Design
We-Design
Dec 11, 2025 · Fundamentals

Why Your A/B Test Results Might Mislead You—and How to Interpret Them Correctly

This article explains the core concepts of A/B testing, including significance, p‑values, minimum sample size, experiment duration, common interpretation pitfalls, and practical e‑commerce conversion tips, helping designers and product teams make data‑driven decisions without falling into statistical traps.

A/B testingdata interpretatione-commerce conversion
0 likes · 18 min read
Why Your A/B Test Results Might Mislead You—and How to Interpret Them Correctly
Data Party THU
Data Party THU
Nov 19, 2025 · Industry Insights

Why Traditional A/B Tests Fail in Two‑Sided Markets—and How to Fix Them

The article examines how conventional single‑sided A/B testing breaks down in two‑sided markets due to SUTVA violations, cross‑interference, and spillover effects, and presents practical mitigation strategies such as small‑world partitioning, counterfactual interleaving, and model‑based corrections.

A/B testingSUTVAcounterfactual interleaving
0 likes · 9 min read
Why Traditional A/B Tests Fail in Two‑Sided Markets—and How to Fix Them
Radish, Keep Going!
Radish, Keep Going!
Oct 18, 2025 · Artificial Intelligence

Gemini 3.0 Unveiled: Google’s AI Leap in Coding and Multimodal Power

Google’s Gemini 3.0, spotted through an A/B test on AI Studio, showcases dramatic improvements in coding precision, SVG generation, and multimodal understanding, offering developers faster UI/UX code, larger output lengths, and higher quality than Gemini 2.5, while community discussions highlight its potential and access challenges.

A/B testingAI modelGemini 3.0
0 likes · 10 min read
Gemini 3.0 Unveiled: Google’s AI Leap in Coding and Multimodal Power
58UXD
58UXD
Sep 9, 2025 · Product Management

How AI-Generated Job Cards Boost Recruitment Click‑Through and Conversion

In a competitive hiring market, the team used AI to redesign multi‑platform job cards, addressing three major pain points—homogeneous information, outdated templates, and misalignment with young users—by creating four visual styles and a standardized generation pipeline, resulting in higher click‑through and application rates.

A/B testingAIJob Card Design
0 likes · 8 min read
How AI-Generated Job Cards Boost Recruitment Click‑Through and Conversion
JD Tech Talk
JD Tech Talk
Aug 27, 2025 · Artificial Intelligence

How AI‑Generated Virtual Try‑On Boosted Fashion Sales by 80%+

This article details JD Retail's AI‑driven virtual try‑on system, covering business challenges, technical hurdles, core algorithmic innovations, practical results from a major fashion brand, and future directions for personalized, automated, and predictive AI try‑on in e‑commerce.

A/B testingAIAIGC
0 likes · 15 min read
How AI‑Generated Virtual Try‑On Boosted Fashion Sales by 80%+
JD Cloud Developers
JD Cloud Developers
Aug 27, 2025 · Artificial Intelligence

How AI Virtual Try‑On Boosted Fashion Sales by 80%: A Technical Deep‑Dive

This article details how JD.com’s AI‑driven virtual fitting solution, integrated with an A/B testing platform, transformed fashion e‑commerce by generating realistic model images and videos, cutting production costs to zero, accelerating design cycles, and increasing conversion rates by over 80% during major sales events.

A/B testingAIFashion E‑commerce
0 likes · 14 min read
How AI Virtual Try‑On Boosted Fashion Sales by 80%: A Technical Deep‑Dive
Mashang Consumer UXC
Mashang Consumer UXC
Aug 21, 2025 · Fundamentals

How to Design Layouts That Readers Instantly Understand

In an age of information overload, this guide explains visual scanning patterns like F‑ and Z‑reading, offers practical layout, typography, and visual‑guidance techniques, and shows how to validate designs with A/B testing, heatmaps, and performance optimization to make content effortlessly readable.

A/B testingUX designheatmap
0 likes · 8 min read
How to Design Layouts That Readers Instantly Understand
JD Tech
JD Tech
Jul 23, 2025 · Artificial Intelligence

AI Virtual Try‑On Transforms Fashion E‑Commerce, Raising Conversion 80%

JD Retail’s “JingDianDian” AI virtual try‑on platform leverages a 12‑billion‑parameter Flux‑Fill diffusion model and multimodal pose estimation to automatically create realistic model images and videos, integrates with the JingMai A/B testing system, and delivers up to an 80% boost in conversion while cutting production costs and time dramatically.

A/B testingAIFashion Tech
0 likes · 13 min read
AI Virtual Try‑On Transforms Fashion E‑Commerce, Raising Conversion 80%
JD Retail Technology
JD Retail Technology
Jul 15, 2025 · Artificial Intelligence

How AI Virtual Try‑On Boosted Fashion Sales by 80%: JD’s Innovative Solution

This article details JD Retail Technology’s AI‑driven virtual try‑on system that combines a 12B Flux‑Fill diffusion model with a high‑quality virtual model library and integrates with the JingMai A/B testing platform, cutting production costs to zero, slashing cycle time to half a day, and increasing order conversion rates by over 80% during the 618 shopping festival.

A/B testingAIFashion E‑commerce
0 likes · 13 min read
How AI Virtual Try‑On Boosted Fashion Sales by 80%: JD’s Innovative Solution
JD Tech Talk
JD Tech Talk
Jul 4, 2025 · Artificial Intelligence

How AI‑Driven Virtual Try‑On Boosted Fashion Sales by 80%

This article details how JD.com’s AI-powered virtual try‑on system, integrated with the Jingmai A/B testing platform, transformed fashion e‑commerce by generating realistic model images and videos, reducing production costs to near zero, cutting design cycles from weeks to hours, and increasing conversion rates by over 80% during major sales events.

A/B testingAIAIGC
0 likes · 14 min read
How AI‑Driven Virtual Try‑On Boosted Fashion Sales by 80%
JD Tech Talk
JD Tech Talk
Jun 12, 2025 · Product Management

How to Tackle Outliers in Internet A/B Experiments: Methods, Pitfalls, and Practical Tips

This article explores why outliers appear in large‑scale internet A/B tests, explains their impact on experiment precision, compares traditional trim and winsorize techniques, reviews a range of statistical and machine‑learning detection methods, and offers practical recommendations for handling them in product experiments.

A/B testingexperiment designoutlier detection
0 likes · 15 min read
How to Tackle Outliers in Internet A/B Experiments: Methods, Pitfalls, and Practical Tips
JD Cloud Developers
JD Cloud Developers
Jun 12, 2025 · Fundamentals

How to Tackle Outliers in Internet A/B Experiments: Methods & Best Practices

This article explores why outliers destabilize online A/B tests, explains their statistical definitions, compares trimming and winsorizing techniques, reviews classic and machine‑learning detection methods, and offers practical guidance for applying these approaches to improve experiment reliability.

A/B testingexperimental designoutlier detection
0 likes · 14 min read
How to Tackle Outliers in Internet A/B Experiments: Methods & Best Practices
Meituan Technology Team
Meituan Technology Team
Jun 5, 2025 · Fundamentals

Unlocking Randomized Experiments: Advanced Techniques to Boost Test Power

This comprehensive guide explores the fundamentals of randomized controlled experiments, discusses classic RCT designs and their limitations, and presents advanced methods such as CUPED variance reduction, stratified, paired, and covariate‑adaptive randomization, as well as spill‑over modeling and random saturation designs to improve experimental power and reliability.

A/B testingCUPEDRandomized Controlled Experiments
0 likes · 59 min read
Unlocking Randomized Experiments: Advanced Techniques to Boost Test Power
Meituan Technology Team
Meituan Technology Team
May 22, 2025 · Fundamentals

Why Write an A/B Experiment Whitepaper? – Overview and Methodology

This whitepaper introduces the importance of data‑driven A/B testing, outlines its theoretical foundations, practical challenges such as small samples and spillover effects, and presents a structured roadmap—including experiment basics, statistical principles, advanced designs, and SDK implementation—to help practitioners design trustworthy experiments.

A/B testingData-drivencausal inference
0 likes · 18 min read
Why Write an A/B Experiment Whitepaper? – Overview and Methodology
ByteDance Data Platform
ByteDance Data Platform
Apr 16, 2025 · Product Management

How A/B Testing Turns Guesswork into Data‑Driven Business Success

In today's fast‑changing market, traditional intuition‑based decisions falter, but systematic A/B testing—illustrated by ByteDance’s academic‑loop culture and real‑world case studies—empowers organizations to replace guesswork with evidence, accelerate innovation, and achieve measurable performance gains across products and strategies.

A/B testingData-drivendecision making
0 likes · 18 min read
How A/B Testing Turns Guesswork into Data‑Driven Business Success
VMIC UED
VMIC UED
Mar 19, 2025 · Product Management

Unlock Higher Conversions: Master the LIFT Model for Landing Page Success

This article explains the LIFT model—a six‑factor framework for landing‑page conversion optimization—detailing its components, a four‑stage workflow from problem identification to result analysis, and practical tips for applying the model to everyday design projects.

A/B testingLIFT modelProduct Design
0 likes · 17 min read
Unlock Higher Conversions: Master the LIFT Model for Landing Page Success
ByteDance Data Platform
ByteDance Data Platform
Feb 12, 2025 · Fundamentals

Why A/B Tests Fail in Recommendation Systems and How to Fix Them

This article examines the hidden complexities of A/B experiments in short‑video recommendation feeds, explains why traditional designs produce biased results due to learning, double‑sided, and network effects, and presents practical double‑sided and community‑randomized experiment frameworks to obtain unbiased strategy evaluations.

A/B testingCommunity randomizationDouble-sided effects
0 likes · 21 min read
Why A/B Tests Fail in Recommendation Systems and How to Fix Them
JD Retail Technology
JD Retail Technology
Jan 7, 2025 · Fundamentals

Handling Outliers in Internet A/B Experiments: Concepts, Methods, and Practical Recommendations

The article explains why outliers destabilize internet A/B tests, outlines their causes, compares trimming and winsorizing, presents lightweight detection (e.g., kurtosis) and risk‑control strategies, and offers practical recommendations for bias‑aware removal and variance‑reduction techniques to improve experimental precision.

.trimA/B testingexperiment design
0 likes · 10 min read
Handling Outliers in Internet A/B Experiments: Concepts, Methods, and Practical Recommendations
Model Perspective
Model Perspective
Dec 2, 2024 · Fundamentals

What Is the Beta Distribution and Why It Matters in A/B Testing?

The Beta distribution is a flexible probability model defined on the interval [0,1] with two shape parameters that control its form, offering useful properties such as mean and variance, and is widely applied in A/B testing, risk assessment, and machine‑learning tasks to model proportions and uncertainties.

A/B testingbeta distributionmachine learning
0 likes · 5 min read
What Is the Beta Distribution and Why It Matters in A/B Testing?
NewBeeNLP
NewBeeNLP
Oct 29, 2024 · Artificial Intelligence

How Hierarchical LLMs Are Transforming Recommendation Systems – A Deep Dive into HLLM

This article provides a comprehensive analysis of the HLLM paper, detailing the motivation behind using large language models for recommendation, the hierarchical architecture of Item and User LLMs, the training objectives, extensive offline and online experiments, scaling behavior, and practical deployment insights.

A/B testingHierarchical LLMLLM for recommendation
0 likes · 12 min read
How Hierarchical LLMs Are Transforming Recommendation Systems – A Deep Dive into HLLM
Meituan Technology Team
Meituan Technology Team
Sep 5, 2024 · Industry Insights

Next‑Generation AB Experiment Analysis Engine for Multi‑Sided Scenarios

The article presents a next‑generation experiment analysis engine that standardizes the core AB testing framework, integrates advanced statistical solutions to tackle small‑sample and overflow challenges, and offers precise variance and P‑value calculations, thereby improving reliability and efficiency for multi‑side fulfillment platform experiments.

A/B testingexperiment analysisfulfillment platform
0 likes · 24 min read
Next‑Generation AB Experiment Analysis Engine for Multi‑Sided Scenarios
ByteDance Data Platform
ByteDance Data Platform
Jul 31, 2024 · Product Management

How Data‑Driven Flywheels Power User Growth: Insights from Volcengine

This article shares a data‑centric perspective on user growth, covering entropy reduction, information management, the data‑driven flywheel, A/B testing practices, retention strategies, and practical case studies that illustrate how systematic data analysis fuels sustainable product expansion.

A/B testingData-drivenEntropy Reduction
0 likes · 16 min read
How Data‑Driven Flywheels Power User Growth: Insights from Volcengine
DataFunTalk
DataFunTalk
Jul 22, 2024 · Fundamentals

A/B Testing and Causal Inference: Evolution of Sampling, Metric Evaluation, and Statistical Inference

The article reviews the development of online A/B testing, covering sampling and traffic‑splitting techniques, metric computation improvements, statistical inference advances, and current challenges such as interference, real‑time inference, and large‑scale metric computation, while referencing recent research papers.

A/B testingMetric EvaluationSampling
0 likes · 10 min read
A/B Testing and Causal Inference: Evolution of Sampling, Metric Evaluation, and Statistical Inference
FunTester
FunTester
Jul 12, 2024 · Product Management

How A/B Testing Can Supercharge Your Product’s Conversion and User Experience

This article explains the fundamentals of A/B testing, outlines its key benefits such as data‑driven decisions, improved user experience and higher conversion rates, and provides practical best‑practice guidelines, essential tools, and required skills for successful implementation.

A/B testingUser experiencebest practices
0 likes · 11 min read
How A/B Testing Can Supercharge Your Product’s Conversion and User Experience
DataFunSummit
DataFunSummit
Jun 22, 2024 · Artificial Intelligence

Applying Causal Inference and Uplift Modeling for User Growth: Concepts, Methods, and Practice

This article introduces causal inference fundamentals, distinguishes correlation from causation, reviews major methodological streams, and demonstrates how uplift and gain models—implemented with T‑learner, S‑learner, and tree‑based approaches—can be applied to user growth and marketing scenarios, including evaluation metrics and future challenges.

A/B testingUplift Modelingcausal inference
0 likes · 14 min read
Applying Causal Inference and Uplift Modeling for User Growth: Concepts, Methods, and Practice
DataFunTalk
DataFunTalk
Jun 13, 2024 · Artificial Intelligence

A/B Testing and Model Grayscale in Credit Risk Control: Concepts, Requirements, and Integrated Solutions

This article explains how A/B testing and model grayscale are applied in credit risk control, discusses the specific requirements for effective testing, compares upstream and risk‑system traffic splitting methods, and proposes an integrated all‑in‑one solution to simplify feature engineering, model evaluation, and deployment.

A/B testingcredit riskfeature engineering
0 likes · 5 min read
A/B Testing and Model Grayscale in Credit Risk Control: Concepts, Requirements, and Integrated Solutions
DataFunTalk
DataFunTalk
May 25, 2024 · Fundamentals

Systematic Solutions to the AA Problem in Random Experiments

This talk explains how combining heavy randomization with regression adjustment can effectively mitigate AA problems in A/B testing, improving experiment credibility by addressing covariate imbalance and enhancing result validity for data‑driven decision making.

A/B testingAA problemData Science
0 likes · 2 min read
Systematic Solutions to the AA Problem in Random Experiments
DataFunTalk
DataFunTalk
May 23, 2024 · Fundamentals

Systematic Solutions to the AA Problem in Random Experiments

Speaker Wanbo Kui, a Didi data analyst, will present a systematic approach to addressing the AA problem in random experiments, covering academic and industry research on re-randomization, its principles and simulations, practical applications, and how it enhances experiment validity.

A/B testingAA problemexperiment design
0 likes · 3 min read
Systematic Solutions to the AA Problem in Random Experiments
DataFunTalk
DataFunTalk
May 22, 2024 · Fundamentals

Systematic Solutions to the AA Problem in Random Experiments

This presentation introduces the AA problem that can compromise A/B test validity and explains how combining re‑randomization with regression adjustment provides an effective, practical solution to improve experiment reliability and credibility.

A/B testingAA problemexperiment design
0 likes · 3 min read
Systematic Solutions to the AA Problem in Random Experiments
DataFunSummit
DataFunSummit
May 12, 2024 · Artificial Intelligence

Pairwise Data Based A/B Experiments: Unbiased Causal Inference in Network Experiments

The DataFun Data Science Summit on May 25 will feature Tencent data scientist Li Yilin presenting a comprehensive talk on pairwise‑data A/B experiments, covering unbiased estimation under various randomizations, theoretical analysis, and practical insights for causal inference in network‑driven online experiments.

A/B testingcausal inferencenetwork experiments
0 likes · 4 min read
Pairwise Data Based A/B Experiments: Unbiased Causal Inference in Network Experiments
DataFunTalk
DataFunTalk
May 12, 2024 · Artificial Intelligence

Paired Data Based A/B Experiments: Causal Inference in Network Experiments

The DataFun Data Science Summit on May 25 will feature Tencent data scientist Li Yilin presenting a comprehensive overview of paired‑data A/B experiments, covering causal inference challenges, unbiased estimators under various randomization designs, theoretical analysis, and practical insights for network‑based online experiments.

A/B testingcausal inferencenetwork experiments
0 likes · 5 min read
Paired Data Based A/B Experiments: Causal Inference in Network Experiments
DataFunSummit
DataFunSummit
May 9, 2024 · Fundamentals

Technical Evolution and Challenges of Online A/B Testing

This article reviews the two‑decade evolution of online A/B testing, outlines the business and technical challenges enterprises face, and details three core technical challenges—experiment accuracy, analysis & interpretation, and efficiency—along with practical solutions for each.

A/B testingAnalysisdata-driven decision
0 likes · 6 min read
Technical Evolution and Challenges of Online A/B Testing
ByteDance Data Platform
ByteDance Data Platform
May 8, 2024 · Backend Development

How DataTester’s Architecture Upgrade Uses DDD to Tame Code Complexity

DataTester’s A/B testing platform underwent a comprehensive architectural overhaul, applying domain‑driven design, modular refactoring, automated validation, and dependency inversion to reduce change amplification, cognitive load, and unknown unknowns, ultimately improving code readability, maintainability, scalability, and development efficiency across its lifecycle.

A/B testingCode RefactoringDDD
0 likes · 29 min read
How DataTester’s Architecture Upgrade Uses DDD to Tame Code Complexity
NetEase Cloud Music Tech Team
NetEase Cloud Music Tech Team
Apr 23, 2024 · Mobile Development

Cloud Music User Push Notification Optimization: Practices and Insights

Cloud Music revamped its push‑notification system by separating business and channel layers, integrating a unified delivery platform, tailoring messages to Android manufacturers, adding new push channels, refining frequency and copy controls, and using AI‑generated creatives, which together doubled click‑through rates and nearly doubled total click users within two months.

A/B testingAIGC Content GenerationAlgorithm Optimization
0 likes · 23 min read
Cloud Music User Push Notification Optimization: Practices and Insights
JD.com Experience Design Center
JD.com Experience Design Center
Apr 19, 2024 · Product Management

How A/B Testing Transforms JD Express Mini‑Program Design: From Basics to Real‑World Results

This article explains why and how to conduct A/B testing for UI design, outlines experiment setup, variable creation, and data analysis, and presents detailed case studies of JD Express mini‑program pop‑up and order‑completion page experiments that demonstrate measurable improvements in click‑through and conversion rates.

A/B testingUX designdata analysis
0 likes · 18 min read
How A/B Testing Transforms JD Express Mini‑Program Design: From Basics to Real‑World Results
DataFunSummit
DataFunSummit
Mar 10, 2024 · Artificial Intelligence

Evaluating Long-Term Strategy Effects with A/B Experiments: Causes, Industry Solutions, and Business Cases

This article examines why A/B experiments often capture only short‑term impacts, explains external and internal factors behind short‑ and long‑term effects, and presents seven industrial methods—including user‑learning models, personalized recommendation adjustments, surrogate metrics, and bias correction—supported by real‑world case studies.

A/B testingBias Correctioncausal inference
0 likes · 14 min read
Evaluating Long-Term Strategy Effects with A/B Experiments: Causes, Industry Solutions, and Business Cases
DataFunTalk
DataFunTalk
Feb 1, 2024 · Fundamentals

Understanding Search Experiments: AB Testing, Experiment Types, and Common Issues

This article explains search experiments from a data‑product viewpoint, covering AB testing fundamentals, multi‑layer experiment architecture, four experiment types (ordinary AB, vocabulary, diff‑AB, interleaving), real‑world case studies, and a comprehensive FAQ addressing typical challenges and troubleshooting methods.

A/B testingData Productalgorithm evaluation
0 likes · 10 min read
Understanding Search Experiments: AB Testing, Experiment Types, and Common Issues
ByteDance Data Platform
ByteDance Data Platform
Jan 31, 2024 · Artificial Intelligence

How A/B Testing Powers Continuous Improvement in Recommendation Systems

This article explains the role of A/B experiments in recommendation systems, outlines their workflow, shares practical tips and parameter design strategies, and demonstrates how to use experiment parameters and feature flags for efficient testing, optimization, and full‑scale deployment.

A/B testingexperiment parametersfeature flag
0 likes · 15 min read
How A/B Testing Powers Continuous Improvement in Recommendation Systems
Model Perspective
Model Perspective
Jan 22, 2024 · Artificial Intelligence

How A/B Testing and the ε‑Greedy Multi‑Armed Bandit Can Boost Decisions

This article explains the principles of A/B testing and the ε‑greedy multi‑armed bandit algorithm, illustrates their practical use in e‑commerce recommendation optimization, and draws broader life lessons about balancing exploration and exploitation for better personal and professional decisions.

A/B testingexploration vs exploitationgreedy
0 likes · 6 min read
How A/B Testing and the ε‑Greedy Multi‑Armed Bandit Can Boost Decisions
StarRocks
StarRocks
Jan 10, 2024 · Big Data

How Tencent Built the ABetterChoice SaaS A/B Testing Platform for Global Games

In 2022 Tencent's A/B Test team created the overseas SaaS product ABetterChoice, abstracting internal experiment capabilities, adapting to multi‑cloud compliance, and unifying computation with StarRocks, enabling game titles like Honor of Kings, PUBG Mobile, and Ubisoft to run scalable, compliant A/B experiments worldwide.

A/B testingData LakeExperiment Platform
0 likes · 14 min read
How Tencent Built the ABetterChoice SaaS A/B Testing Platform for Global Games
Huolala Tech
Huolala Tech
Dec 29, 2023 · Fundamentals

How Variance Reduction Boosts A/B Test Sensitivity Without More Samples

This article explains why variance reduction is essential for A/B experiments, describes at‑assignment and post‑assignment techniques such as stratified sampling, post‑stratification and CUPED, compares their effectiveness, and presents real‑world case studies demonstrating how they improve experiment sensitivity without increasing sample size.

A/B testingCUPEDexperiment sensitivity
0 likes · 13 min read
How Variance Reduction Boosts A/B Test Sensitivity Without More Samples
DataFunTalk
DataFunTalk
Dec 20, 2023 · Fundamentals

Evaluating Long-Term Effects of Strategies with A/B Experiments: Methods and Case Studies

This article examines why A/B experiments often capture only short‑term impacts, categorises external and internal causes of short‑term bias, and presents seven industry‑tested approaches—including user‑learning models, personalized recommendation adjustments, surrogate metrics, and bias correction techniques—to reliably estimate long‑term strategy effectiveness, illustrated with real business cases.

A/B testingcausal inferenceexperiment design
0 likes · 13 min read
Evaluating Long-Term Effects of Strategies with A/B Experiments: Methods and Case Studies
DataFunTalk
DataFunTalk
Dec 14, 2023 · Fundamentals

Evaluating Long-Term vs Short-Term Effects in A/B Experiments

While A/B testing is widely used for data-driven decisions, short-term experimental results often diverge from long-term impacts, leading to misguided strategies; this article examines why such inconsistencies arise and reviews major methods—including extended experiments, holdout groups, post‑analysis, CCD, and surrogate‑metric modeling—to reliably estimate long‑term effects.

A/B testingData ScienceLong-term impact
0 likes · 13 min read
Evaluating Long-Term vs Short-Term Effects in A/B Experiments
Huolala Tech
Huolala Tech
Dec 8, 2023 · R&D Management

How Multi‑Time‑Slice Experiments Boost Traffic Homogeneity and Reduce Bias

This article explains how Huolala's data‑science team tackles interference between multiple time‑slice experiments by using city‑level isolation, nested experiment planning, and bias‑variance trade‑offs, providing detailed guidelines, recovery cycles, and case studies to maximize traffic utilization and experimental reliability.

A/B testingbias‑varianceexperiment design
0 likes · 11 min read
How Multi‑Time‑Slice Experiments Boost Traffic Homogeneity and Reduce Bias
Data Thinking Notes
Data Thinking Notes
Nov 30, 2023 · Product Management

Building a Metric System for Sustainable Growth: From Data to Action

This article explains how to construct a metric system, identify bottlenecks, and design data‑driven growth strategies using Volcano Engine's DataFinder and DataTester, illustrated with real‑world case studies and step‑by‑step A/B testing practices.

A/B testinggrowth metricsproduct optimization
0 likes · 15 min read
Building a Metric System for Sustainable Growth: From Data to Action
Huolala Tech
Huolala Tech
Nov 10, 2023 · Product Management

Mastering A/B Testing in Two‑Sided Markets: Principles, Cases, and Strategies

This article explains how to design and implement A/B experiments in complex two‑sided markets, covering core concepts of causality, detailed case studies, various allocation principles, risk‑benefit trade‑offs, and practical guidelines for selecting appropriate experimental methods across different business scenarios.

A/B testingcausalityexperiment design
0 likes · 9 min read
Mastering A/B Testing in Two‑Sided Markets: Principles, Cases, and Strategies
Test Development Learning Exchange
Test Development Learning Exchange
Oct 28, 2023 · Databases

How Data Analysis Improves User Experience: Methods and Practical SQL Code Examples

This article explains ten data‑analysis techniques for enhancing user experience—such as behavior tracking, A/B testing, sentiment analysis, and personalization—and provides concrete SQL code snippets that illustrate how to import, query, filter, sort, aggregate, join, update, delete, and back up data in relational databases.

A/B testingUser experiencedata analysis
0 likes · 8 min read
How Data Analysis Improves User Experience: Methods and Practical SQL Code Examples
WeChat Backend Team
WeChat Backend Team
Oct 25, 2023 · Fundamentals

Mastering Metric Covariance for Accurate A/B Test Analysis

This article explains the statistical foundations of A/B testing, introduces potential outcomes and average treatment effect, defines metric covariance, and presents practical estimation methods—including naive, data‑augmentation, and bucket‑based approaches—along with real‑world performance evaluations and applications such as variance reduction and Bayesian optimization.

A/B testingBayesian Optimizationexperimental design
0 likes · 18 min read
Mastering Metric Covariance for Accurate A/B Test Analysis
DataFunSummit
DataFunSummit
Oct 17, 2023 · Artificial Intelligence

DataFunSummit2023: Deep Learning‑Driven Multi‑Experiment Causal Inference and Distributed Causal Tools

The DataFunSummit2023 online conference brings together experts from Tencent and Kuaishou to present cutting‑edge research on causal inference for large‑scale A/B testing, including deep‑learning‑based multi‑experiment effect estimation, a distributed causal inference framework (Fast‑Causal‑Inference), and strategies for evaluating long‑term policy impacts.

A/B testingData ScienceDeep Learning
0 likes · 7 min read
DataFunSummit2023: Deep Learning‑Driven Multi‑Experiment Causal Inference and Distributed Causal Tools
Architect
Architect
Oct 14, 2023 · Industry Insights

How to Build a Trustworthy A/B Testing Platform for Complex Multi‑Side Marketplaces

This article explains how Meituan's fulfillment team designs, implements, and operates a reliable A/B testing platform for multi‑side markets, detailing statistical pitfalls, experiment types, traffic-splitting frameworks, and automated analysis pipelines to ensure credible results despite overflow effects, small samples, and fairness constraints.

A/B testingexperiment designmulti‑side marketplace
0 likes · 40 min read
How to Build a Trustworthy A/B Testing Platform for Complex Multi‑Side Marketplaces
DevOps Cloud Academy
DevOps Cloud Academy
Oct 11, 2023 · Cloud Native

A/B Testing with Argo Rollouts Experiments for Progressive Delivery

This article explains how to perform data‑driven A/B testing in progressive delivery using Argo Rollouts Experiments, covering the concepts of progressive delivery, A/B testing fundamentals, the Argo Rollouts architecture, required Kubernetes resources, and step‑by‑step commands and YAML manifests for a weather‑app example.

A/B testingArgo RolloutsKubernetes
0 likes · 19 min read
A/B Testing with Argo Rollouts Experiments for Progressive Delivery
DataFunTalk
DataFunTalk
Oct 4, 2023 · Big Data

Insights into Regional Differences in Overseas A/B Experiments

The presentation explains how to detect, analyze, and leverage regional variations in overseas A/B test results to make more informed product decisions, using a systematic experimental analysis framework grounded in causal inference and online experimentation methods.

A/B testingcausal inferencegame data science
0 likes · 2 min read
Insights into Regional Differences in Overseas A/B Experiments
DataFunTalk
DataFunTalk
Sep 30, 2023 · Fundamentals

Different Types of Experiments in Search Scenarios

The presentation by Tencent PCG data product manager Wang Dongxing introduces A/B testing fundamentals and shares practical experiences with various search experiment methods—including regular A/B, vocabulary, diffAB, and interleaving—while highlighting common pitfalls and offering actionable insights for practitioners.

A/B testingData Productonline testing
0 likes · 2 min read
Different Types of Experiments in Search Scenarios
Mashang Consumer UXC
Mashang Consumer UXC
Sep 28, 2023 · Product Management

How Data‑Driven Design Boosted App Popup Click‑Through Rates

This article examines how systematic data analysis and targeted design experiments—covering color, copy, benefit points, button style, and seasonal skins—significantly increased click‑through rates for app pop‑up resources, offering actionable insights for product managers seeking data‑driven UX improvements.

A/B testingUser experienceclick-through rate
0 likes · 6 min read
How Data‑Driven Design Boosted App Popup Click‑Through Rates
DeWu Technology
DeWu Technology
Sep 6, 2023 · Industry Insights

From Simple Split Tests to Real‑Time Multi‑Layer Experiments: The Evolution of an AB Testing Platform

This article traces the step‑by‑step evolution of an AB testing platform—from its initial 1.0 version with basic traffic splitting, through the 2.0 era that introduced multi‑layer orthogonal traffic models and real‑time metric pipelines, to the 3.0 era focused on usability, stability, and advanced analysis—while sharing concrete design decisions, implementation details, and lessons learned.

A/B testingExperiment PlatformReal-time analytics
0 likes · 25 min read
From Simple Split Tests to Real‑Time Multi‑Layer Experiments: The Evolution of an AB Testing Platform
Meituan Technology Team
Meituan Technology Team
Aug 24, 2023 · Product Management

How to Build a Trustworthy A/B Testing Platform for Complex Fulfillment Scenarios

This article presents a comprehensive guide to designing, implementing, and analyzing a reliable A/B testing platform for Meituan's multi‑side fulfillment business, covering statistical pitfalls, experiment types, traffic‑splitting frameworks, automated analysis engines, and practical solutions for overflow effects, small samples, and fairness constraints.

A/B testingexperiment designfulfillment
0 likes · 39 min read
How to Build a Trustworthy A/B Testing Platform for Complex Fulfillment Scenarios
Top Architect
Top Architect
Aug 24, 2023 · Operations

Blue‑Green, Rolling, and Canary Deployment Strategies Overview

This article explains three common software release strategies—blue‑green deployment, rolling deployment, and canary (gray) deployment—detailing their principles, advantages, potential pitfalls, and practical considerations, while also contrasting them with A/B testing and noting related operational concerns.

A/B testingBlue-GreenCanary
0 likes · 12 min read
Blue‑Green, Rolling, and Canary Deployment Strategies Overview
ByteDance Data Platform
ByteDance Data Platform
Aug 16, 2023 · Operations

How LeKe Scaled 1,200 Gyms Using Data‑Driven Ops, Agile Testing & AI Recommendations

In an interview with LeKe CTO Chengshi, the company’s rapid growth is attributed to three data‑powered capabilities—fine‑grained user operation, agile A/B‑testing, and AI‑driven personalized recommendation—enabled by Volcano Engine’s data platform and the data‑flywheel concept.

A/B testingDigital Transformationdata-driven operations
0 likes · 13 min read
How LeKe Scaled 1,200 Gyms Using Data‑Driven Ops, Agile Testing & AI Recommendations
Baidu MEUX
Baidu MEUX
Aug 10, 2023 · Product Management

How Baidu’s “Ask‑One‑Ask” Redefined Service Design to Boost Conversion and Revenue

This case study details Baidu’s systematic redesign of its real‑time Q&A service, covering pre‑service question‑page enhancements, in‑service IM page expansion, and post‑service repurchase guidance, illustrating how iterative, data‑driven design lifted user experience and doubled revenue streams.

A/B testingUser experienceconversion optimization
0 likes · 9 min read
How Baidu’s “Ask‑One‑Ask” Redefined Service Design to Boost Conversion and Revenue
DataFunTalk
DataFunTalk
Aug 8, 2023 · Product Management

ByteDance's A/B Testing Practices: Methodology, Platform, and Real‑World Cases

This article explains why A/B testing is considered the gold standard for causal inference, shares ByteDance’s extensive internal experimentation practices, describes the Volcano Engine platform architecture, outlines how to design and run experiments, and provides real case studies and Q&A for product teams.

A/B testingByteDanceData-driven
0 likes · 25 min read
ByteDance's A/B Testing Practices: Methodology, Platform, and Real‑World Cases
DataFunSummit
DataFunSummit
Aug 6, 2023 · Product Management

ByteDance’s A/B Testing Practices: Theory, Cases, and Platform Overview

This article explains why A/B testing is considered the gold standard for causal inference, shares ByteDance’s extensive internal experimentation practices and case studies, describes the Volcano Engine experiment platform architecture, and outlines the step‑by‑step process for launching reliable A/B experiments.

A/B testingByteDanceData‑Driven
0 likes · 26 min read
ByteDance’s A/B Testing Practices: Theory, Cases, and Platform Overview
Architect
Architect
Aug 5, 2023 · Artificial Intelligence

Architecture and Evolution of a Game Recommendation System

From its inception as a simple game distribution platform to a sophisticated, multi‑layered recommendation architecture, this article details the background, early models, business growth, architectural evolution, caching strategies, GC optimization, rate limiting, experiment platform, multi‑path recall, dynamic tuning, and future intelligent enhancements of a game recommendation system.

A/B testingArtificial IntelligenceGarbage Collection
0 likes · 17 min read
Architecture and Evolution of a Game Recommendation System
DataFunTalk
DataFunTalk
Aug 3, 2023 · Game Development

Applying A/B Testing to Drive Growth in Tencent Overseas Games

This article explains how Tencent leverages A/B testing across its overseas games, detailing market differences, experimental methodology, multi‑cloud platform compliance, data architecture, and case studies that illustrate how targeted experiments improve user onboarding, gameplay settings, and email‑based re‑engagement.

A/B testingGame Analyticsdata pipelines
0 likes · 12 min read
Applying A/B Testing to Drive Growth in Tencent Overseas Games
DataFunSummit
DataFunSummit
Jul 16, 2023 · Game Development

Applying A/B Testing to Drive Growth in Tencent’s Overseas Games

This article explains how Tencent leverages A/B testing across its overseas games, detailing the current market situation, experimental capabilities, multi‑cloud platform architecture, and case studies that illustrate how data‑driven experiments improve user retention, engagement, and overall business growth.

A/B testingData ScienceGame Development
0 likes · 12 min read
Applying A/B Testing to Drive Growth in Tencent’s Overseas Games
Architect's Guide
Architect's Guide
Jul 13, 2023 · Operations

Blue‑Green, Rolling, and Canary Deployment Strategies Explained

This article introduces common release strategies—blue‑green deployment, rolling updates, and canary (gray) releases—detailing their workflows, advantages, drawbacks, and practical considerations, and clarifies how they differ from A/B testing in modern software delivery.

A/B testingBlue-GreenCanary
0 likes · 9 min read
Blue‑Green, Rolling, and Canary Deployment Strategies Explained
DevOps Cloud Academy
DevOps Cloud Academy
Jul 11, 2023 · Cloud Native

A/B Testing with Argo Rollouts Experiments for Progressive Delivery

This article explains how to perform data‑driven A/B testing in progressive delivery using Argo Rollouts Experiments on Kubernetes, covering the concepts, required resources, YAML manifests, command‑line steps, and the benefits of combining A/B tests with canary deployments.

A/B testingArgo RolloutsKubernetes
0 likes · 17 min read
A/B Testing with Argo Rollouts Experiments for Progressive Delivery
DataFunSummit
DataFunSummit
Jul 10, 2023 · Artificial Intelligence

Applying Causal Inference to Business Improvement: Concepts, Methods, and Case Studies from Xiaohongshu

This article explains why causal inference is needed in data‑driven businesses, introduces its theoretical foundations from computer science, econometrics and statistics, and demonstrates how various causal modeling techniques can be used to boost user retention and content creation on the Xiaohongshu platform.

A/B testingBusiness Analyticscausal inference
0 likes · 12 min read
Applying Causal Inference to Business Improvement: Concepts, Methods, and Case Studies from Xiaohongshu
DaTaobao Tech
DaTaobao Tech
Jun 19, 2023 · Product Management

User Experience Analysis of Taobao Detail Page Using User Journey and VOC Data

The article, the second in a ten‑part Taobao APP UX series, explains how module‑level user‑journey metrics and Voice‑of‑Customer chat data are collected, labeled with a BIO‑CRF taxonomy, clustered via DBSCAN, and correlated to identify size and quality concerns on the men’s‑clothing detail page, prompting module redesigns, A/B tests, and resulting in higher conversion rates and reduced dwell time.

A/B testingBig DataUser experience
0 likes · 11 min read
User Experience Analysis of Taobao Detail Page Using User Journey and VOC Data
DevOps
DevOps
Jun 1, 2023 · Operations

Feature Toggles: Concepts, Types, Implementation, and Best Practices

This article explains feature toggles (feature flags), compares them with feature branches, outlines their advantages and disadvantages, describes different toggle types and lifecycles, provides implementation details with code examples, lists open‑source frameworks, presents real‑world case studies, and offers practical guidance for using toggles in modern DevOps workflows.

A/B testingContinuous Deliveryfeature toggle
0 likes · 21 min read
Feature Toggles: Concepts, Types, Implementation, and Best Practices
DataFunTalk
DataFunTalk
Apr 21, 2023 · Product Management

Best Practices for A/B Testing Platforms: Business Applicability, Internal Use Cases, Industry Examples, and Sustainable Experiment Culture

This article presents a comprehensive guide to A/B testing platforms, covering their business applicability, internal implementations at ByteDance, industry-specific case studies, platform architecture, experiment types, and strategies for building a sustainable experiment culture within organizations.

A/B testingCase StudiesData-driven
0 likes · 17 min read
Best Practices for A/B Testing Platforms: Business Applicability, Internal Use Cases, Industry Examples, and Sustainable Experiment Culture
Su San Talks Tech
Su San Talks Tech
Apr 18, 2023 · Backend Development

Mastering Alibaba Sentinel: Real‑Time Flow Control and Circuit Breaking for Spring Cloud

This article introduces Alibaba Sentinel's lightweight traffic‑control and circuit‑breaking capabilities, walks through quick setup, advanced rule configuration, Spring Cloud Alibaba integration, custom extensions, distributed lock and A/B testing implementations, and discusses its limitations and future prospects for microservice reliability.

A/B testingCircuit BreakingSpring Cloud
0 likes · 19 min read
Mastering Alibaba Sentinel: Real‑Time Flow Control and Circuit Breaking for Spring Cloud
DataFunSummit
DataFunSummit
Apr 12, 2023 · Product Management

Best Practices for A/B Testing Platforms: Business Applicability, Internal & Industry Cases, and Sustainable Experiment Culture

This article presents a comprehensive guide to A/B testing platforms, covering their business applicability, internal and external use cases across industries, detailed platform architecture, experiment types, statistical reporting, analysis tools, feature flag management, and recommendations for building a sustainable experiment culture within organizations.

A/B testingData-drivenExperiment Platform
0 likes · 19 min read
Best Practices for A/B Testing Platforms: Business Applicability, Internal & Industry Cases, and Sustainable Experiment Culture
DataFunSummit
DataFunSummit
Apr 6, 2023 · Game Development

Experiment-Driven Advertising and User Operations in Game Growth: Causal Inference, Uplift Modeling, and Practical Pitfalls

This article presents a data‑science‑focused guide on using causal inference and uplift models to drive overseas ad targeting and user‑operation decisions in games, covering audience selection, privacy‑aware exposure correction, bid optimization, experiment design pitfalls, network effects, and practical recommendations.

A/B testingAdvertisingUplift Modeling
0 likes · 18 min read
Experiment-Driven Advertising and User Operations in Game Growth: Causal Inference, Uplift Modeling, and Practical Pitfalls
Tencent Advertising Technology
Tencent Advertising Technology
Mar 28, 2023 · Operations

Experimental Design for Two-Sided Markets in Advertising Scenarios

This article discusses experimental design challenges in two-sided markets, particularly in advertising scenarios, and presents various methods including four-table experiments, counterfactual interleaving, and contingency table joint sampling to address issues like network effects and competition between supply and demand sides.

A/B testingAdvertisingcontingency table sampling
0 likes · 14 min read
Experimental Design for Two-Sided Markets in Advertising Scenarios
DataFunTalk
DataFunTalk
Mar 27, 2023 · Product Management

Metric System Analysis and Growth Practice: Building Indicator Systems, Designing Optimization Strategies, and Real-World Case Studies

This presentation explains how to construct a metric system, identify bottlenecks, design targeted growth strategies, and apply A/B testing through detailed examples from Volcengine's DataFinder and DataTester platforms, culminating in practical case studies from Douyin Group and other enterprises.

A/B testingIndicator Systemcase study
0 likes · 15 min read
Metric System Analysis and Growth Practice: Building Indicator Systems, Designing Optimization Strategies, and Real-World Case Studies
DaTaobao Tech
DaTaobao Tech
Mar 24, 2023 · Industry Insights

How Taobao’s “Home‑Page as Venue” Redefined Big‑Sale Performance

This article details the technical design, challenges, and results of Taobao’s “Home‑Page as Venue” initiative, explaining how a new A/B‑page architecture replaced crowded promotional cards with a dynamic cat‑head entry, improved information‑flow efficiency, reduced development cost, and delivered measurable gains in traffic, conversion, and revenue during major sales events.

A/B testingE‑commerceTaobao
0 likes · 23 min read
How Taobao’s “Home‑Page as Venue” Redefined Big‑Sale Performance
Python Programming Learning Circle
Python Programming Learning Circle
Mar 22, 2023 · Fundamentals

Comparing Distributions Between Groups: Visualization and Statistical Methods in Python

This article demonstrates how to compare the distribution of a variable across control and treatment groups using Python, covering data generation, visual techniques such as boxplots, histograms, KDE, CDF, QQ and ridgeline plots, and a suite of statistical tests including t‑test, SMD, Mann‑Whitney, permutation, chi‑square, Kolmogorov‑Smirnov and ANOVA for both two‑group and multi‑group scenarios.

A/B testingPythonStatistical Tests
0 likes · 21 min read
Comparing Distributions Between Groups: Visualization and Statistical Methods in Python