This article breaks down operational design into three key questions—design basis, user connection, and result validation—using real‑world case studies, visual analysis, and AB testing data to show how designers can create evidence‑driven, conversion‑focused experiences.
The paper presents a WeChat recommendation system that estimates user interest tags via multi‑class classification, using hierarchical intra‑ and inter‑domain attention and dense feature‑crossing to capture diverse preferences, aggregates click‑tag preferences rather than treating all tags equally, and demonstrates superior offline and online performance over baselines such as YouTube‑DNN, AFM, NFM, DCN, and AUTOINT.
This article explains how data analysts can use experimental and observational research methods, including randomized controlled trials, quasi‑experiments, difference‑in‑differences, regression discontinuity, synthetic control, and Bayesian structural time‑series, to assess the causal impact of product and marketing changes on business metrics.
This article presents a three‑stage, data‑driven framework for online driver repositioning that generates candidate dispatch tasks, scores them using a marginal gain model, and selects optimal tasks via a minimum‑cost flow planning algorithm, demonstrating significant improvements in driver efficiency and experience through large‑scale A/B experiments.
The paper describes Didi’s online driver‑repositioning system that treats idle‑driver dispatch as a recommender problem, generating candidate destinations, scoring tasks with a marginal‑gain model, and selecting optimal assignments via a minimum‑cost‑flow optimizer, which in live A/B tests boosted driver efficiency, earnings, and satisfaction while reducing empty cruising.
This comprehensive guide explains AB testing fundamentals—from defining control and experimental groups and avoiding confounding factors, to calculating sample size, selecting ratio‑based metrics, tracking data, monitoring experiments, and analyzing statistical significance—providing a step‑by‑step methodology for data‑driven product optimization.
This article introduces Didi's adaptive grouping algorithm, which enhances the uniformity of user allocation in AB experiments by replacing traditional complete randomization with a single-pass method that balances observed metrics across groups, and demonstrates its effectiveness through large‑scale experimental results.
Didi’s AI Lab introduces an Adaptive grouping algorithm for its Apollo AB‑testing platform that allocates users in a single pass using direct and indirect scores, achieving over 95 % balance probability and reducing group imbalance from 14 % (CR) and 2.7 % (RR) to under 0.8 %.
The article details Ctrip's challenges with multiple AB testing splitters, presents performance gains after migrating to a new splitter, and explains the comprehensive redesign covering overall architecture, interface consolidation, SDK slimming, and a custom distributed cache backend to achieve higher throughput and lower latency.
This article explains the evolution, responsibilities, and evaluation methods of the product‑effect assessment team at Amap, covering offline testing, AB experiments, metric analysis, automated scoring models, and the tools that support a comprehensive product‑performance framework.
This article outlines a comprehensive, standardized front‑end AB testing architecture that separates experiment configuration and data chains, introduces a JSSDK with Core and Coupler packages, and explains traffic‑splitting models, data back‑flow, and extensibility across multiple front‑end DSLs.
This article describes how 58 Tongzhen leverages AI technologies—including data pipelines, feature engineering, various recall and ranking models, and AB‑testing—to build a personalized feed recommendation system for the down‑market, detailing its overall architecture, data sources, model iterations, performance gains, and future directions.
The article explains how Facebook manages dynamic runtime configuration for millions of services—covering feature gating, experiments, traffic control, topology balancing, monitoring, machine‑learning model updates, and internal behavior—using a suite of tools such as Configerator, Gatekeeper, Package Vessel, Sitevars, and MobileConfig.
The article details the design principles, modular architecture, and engineering optimizations of a backend algorithm platform that uses APIs, micro‑services, and asynchronous processing to enable fast, reliable, and scalable online algorithm services, including recall, ranking, metadata, feature reporting, and A/B testing.
This article presents a comprehensive overview of the Automobile Home recommendation system, detailing its objectives, architecture, various ranking models from LR to DeepFM, online learning mechanisms, service APIs, feature engineering pipelines, model training platforms, debugging tools, and future optimization directions.
This article explains how the Nanomirror data analysis platform introduces intelligent slice analysis—both activity‑metric and AB‑bucket slice analyses—to uncover the most impactful user segments, guide targeted interventions, and improve operational efficiency for Xianyu’s online campaigns.
The article details the five‑year evolution of Autohome's recommendation system, covering its overall architecture, the progression of ranking models from LR to DeepFM and online learning, feature engineering pipelines, ranking service APIs, AB testing practices, and future optimization directions.
By extracting hour‑level activity logs and training supervised models—including CART, GBDT, and neural networks—on user tags, the team simulated short‑term metrics for new reward campaigns, enabling earlier prediction of next‑day retention and shortening experiment cycles despite delayed T+1 data.
Meituan’s Turing platform consolidates the entire delivery‑order workflow—from massive data ingestion and feature generation to model training, evaluation, deployment, real‑time prediction, and AB testing—into a single, end‑to‑end system that evolved from a minimal MVP into a fully platformized solution, addressing speed, accuracy, and engineering‑algorithm decoupling while planning deeper deep‑learning integration.
The article outlines a micro‑service based intelligent recommendation platform that supports over 40 scenarios, detailing its overall architecture, AB testing service, and the three core modules—index, recall, and filter—while also describing future plans for platform centralization and open development.
Youzan’s big‑data team boosted SparkSQL stability and performance by reinforcing the Thrift Server, implementing AB gray‑release testing, collecting real‑time metrics, adding an engine‑selection service, and completing a second migration that raised SparkSQL’s workload share to 91 %, while documenting key pitfalls and tuning lessons.
Wedge is a Meituan‑Dianping advertising experiment configuration platform that provides extensible, flow‑based A/B testing with version control, real‑time monitoring, and a user‑friendly UI, enabling algorithm, engineering, and business teams to rapidly iterate, audit, and roll back complex vertical and horizontal experiments.
This article presents a comprehensive case study of how 58.com built a personalized recommendation system for its large‑scale recruitment platform, covering business background, data challenges, user modeling, recall strategies, ranking pipelines, system architecture, experimental infrastructure, and future research directions.
This article demonstrates how OpenResty can be used in production to implement static and dynamic blacklists, request rate limiting, AB testing, and service quality monitoring by embedding Lua scripts into Nginx, with detailed configuration examples and code snippets.
This article details the design goals, architecture, core functionalities, and optimization strategies of Ctrip's user precise marketing system, which leverages RESTful integration, flexible rule-based and machine‑learning models, real‑time monitoring, and AB testing to improve traffic utilization and conversion rates.
The Xianyu team built a flexible gameplay engine that lets product define task‑based challenges, run bucket‑AB tests, and provide real‑time rewards, boosting user participation to over 70% and increasing activity by 30%, while enabling rapid iteration and future expansion across the app and other services.
This article explains the fundamentals of AB testing, outlines the iterative workflow, shares best‑practice guidelines, compares single‑layer and multi‑layer experiment frameworks, and details the technical implementation—including SDK design, hashing algorithms, data denoising, and statistical evaluation methods.
This article explains fundamental data‑analysis methods, introduces the AARRR user‑growth model with key metrics for each stage, and presents practical tools such as user path analysis, funnel conversion, heatmaps, and A/B testing to help product teams make data‑driven decisions and continuously improve user experience.
The article details the design principles, layered traffic allocation model, architecture, data processing pipeline, and operational experience of the AB experiment platform used at Beike Zhaofang, highlighting its web, API, and storage layers, gray‑release capabilities, current limitations, and future improvements.
The article details how 58.com’s Cupid push control system leverages machine‑learning models, especially XGBoost‑based CTR prediction, to prioritize and filter billions of daily push notifications, improving click‑through rates, reducing user annoyance, and providing a scalable, data‑driven architecture for diverse business services.
This article outlines Baidu's comprehensive product evaluation framework, describing its multi‑layer assessment system, the combination of subjective and objective metrics, and a suite of common evaluation methods such as indicator analysis, AB testing, user feedback, behavior analysis, big‑data profiling, and competitor comparison.
This article explains Hulu's comprehensive approach to streaming quality optimization, covering the video system architecture, business model, the distinction between QoS and QoE, key performance metrics, adaptive bitrate algorithms, data‑driven workflows, offline validation, online A/B testing, and the measurable improvements achieved across multiple platforms.
This article explains the fundamentals of Bayes' theorem, shows how to compute prior, likelihood, and posterior probabilities, demonstrates Bayesian A/B testing with Python code, introduces Bayesian networks for causal inference, and discusses the role of Bayesian methods in machine learning and data‑driven decision making.
The article explains the core concepts of recommendation engines—user profiling and data classification—describes how large‑scale data processing tools are used to build models, and outlines common offline and A/B testing approaches for evaluating recommendation performance.
The article provides a comprehensive overview of KDD 2017, including acceptance statistics, best paper awards, Ant Group's contributions, detailed discussions on AB testing, graph mining, and selected research papers across data mining, machine learning, and anomaly detection, offering valuable insights for practitioners and researchers.
This article explains the challenges of testing machine‑learning models for strategy projects, outlines the overall testing workflow, describes key offline and online evaluation metrics such as AUC and AB‑testing, and summarizes best‑practice procedures for assessing model performance, user experience, and effect differences.
The article details how Ctrip’s CTO Gan Quan has leveraged big‑data platforms, deep‑learning algorithms, cross‑screen user tracking, and rapid AB testing to create a real‑time, personalized recommendation engine that shortens travel decision cycles and drives significant revenue growth.
This article explains how subtle AB‑test variations—such as rounding prices or adjusting font size—can dramatically affect purchase intent and comment participation, backed by two real‑world case studies and practical takeaways for designers and product managers.
The article details how Youku Tudou redesigned its video recommendation architecture, moving ranking from offline to online processing, outlining the comparative architecture, benefits, challenges, feature handling, offline evaluation methods, and weight‑fusion techniques that enabled a successful launch after two months of development.
The Youku Tudou data team overhauled its video recommendation system by moving ranking from offline to online, detailing architectural changes, advantages, challenges, feature handling, offline evaluation, and model weight fusion to improve scalability and user experience.
The article explains how Ctrip’s flight‑booking team uses AB testing—from definition and experimental design to data collection, traffic allocation, orthogonal experiments, and result analysis—to drive conversion‑rate and revenue improvements across multiple platforms.
A ByteDance product manager shares how rapid company growth, data‑driven decision making, AB testing, and close collaboration with reliable engineering teams accelerate personal development and create ample space for product innovation.
The article shares practical lessons from a year of backend R&D experiments at Weibo, describing how a bottom‑up hybrid‑cloud solution, cross‑team collaboration, and systematic team‑building transformed the Feed service’s scalability, cost efficiency, and alignment with business goals.
The eight‑hour technical session at Ctrip’s second investment summit in Shanghai featured a series of expert talks covering infrastructure scaling, automation frameworks, call‑center efficiency, secure development lifecycles, pricing engine architecture, high‑availability engineering, AB testing strategies, and mobile app componentization, providing valuable insights for large‑scale internet companies.
