How to Build a Scalable Frontend A/B Testing Framework

Design Approach

Previously, front‑end teams had to write custom logic for each A/B test, leading to high effort, limited coverage across platforms, and coarse‑grained data. A standardized, simple, and efficient A/B testing chain was created to address these pain points.

Experiment Chain Overview

A complete A/B testing chain consists of two parts: the experiment configuration link (creation, saving, grouping, publishing, and real‑time traffic splitting) and the experiment data link (data tagging, filtering, calculation, and reporting). Combining them forms a closed loop.

Experiment Configuration Model

Configurations are grouped by application and pushed to a CDN. At runtime the front‑end reads the configuration via a JSSDK, performs traffic splitting, and renders the appropriate component.

An application can be a mini‑program, a specific page, or any front‑end project. Within an application, a scene represents an independent traffic entry (e.g., the home page or a navigation flow) and serves as the underlying traffic‑splitting model.

Experiment Data Model

Business domains consist of a set of data metrics. Experiments under the same business can be linked to all metrics in that domain, and the metrics are collected, computed, and displayed in experiment reports.

For example, in the “Taobao Coins” business, an experiment may compare two marketing modules on the home scene, using exposure UV and click UV as metrics. Business can span multiple applications, so separating application from business allows shared metrics across apps.

Design Solution

The front‑end A/B testing design aligns the configuration and data models into a cohesive platform. New business units first register a workspace consisting of an application (for configuration), a business (for metrics), and a scene (traffic model).

After workspace creation, the core experiment creation flow includes reading basic info, setting traffic buckets, and automatically generating front‑end integration code.

Experiments can be published, with an optional beta release for pre‑validation. Multiple releases are supported, allowing traffic re‑allocation before final launch.

Runtime – JSSDK

The JSSDK runs in the front‑end project, reads experiment configuration, performs real‑time splitting, and returns the component to render. It also reports split results and metric data back to the platform.

The JSSDK is split into two packages:

Core – handles configuration fetching, caching, experiment lifecycle, and splitting algorithms.

Coupler – a DSL‑specific adapter (e.g., React, Mini‑Program) that implements a defined interface for storage, cookies, and other environment‑specific APIs, keeping Core completely decoupled from any front‑end framework.

Core first checks a version‑control strategy to fetch updated configuration only when needed, then validates the experiment period before invoking the split algorithm. The Coupler uses the split result to render the appropriate component and reports the result to the platform. An auto‑generated tracking component embeds the metric‑reporting code for each experiment.

Extension Plan

Because the Core is framework‑agnostic, extending the JSSDK to new DSLs (React, Mini‑Program, Node FaaS, etc.) only requires implementing the Coupler according to the interface specification, ensuring low‑cost scalability and maintainability.

Traffic Splitting Model

Within a scene, experiments are organized into layers . Layers are orthogonal (traffic can flow through multiple layers), while experiments inside the same layer are mutually exclusive. A special Launch Layer holds experiments that have been rolled out to full traffic, bypassing the split algorithm.

JSSDK Traffic Splitting

The split factor is the browser cookie field

cna

, which uniquely identifies a web device. This works across web, mobile, and PC, but may produce different groups for the same user on different devices. The design allows future extensions to support user‑id, utdid, or custom split factors.

Data Backflow

Each newly created metric receives a unique UID. When a component is rendered, the JSSDK reports the split result and metric data, including a log key, experiment publish ID, group ID, and metric UID. The platform runs scheduled jobs to filter experiment logs by the log key, aggregates UV/PV per metric, and generates the final experiment report.

Conclusion

Front‑end A/B testing remains an under‑exploited area. By providing a standardized, data‑driven experimentation platform, engineers can validate each iteration scientifically and make decisions based on real metrics rather than intuition.