How Alibaba Masters Full‑Chain Performance Testing for Double 11

Preface

Performance testing is essential for large‑scale marketing events to mitigate system uncertainty caused by traffic surges. Alibaba presents a complete full‑chain testing workflow that covers environment preparation, data setup, traffic safety, execution, and analysis, helping enterprises achieve reliable performance during peak periods.

Process Overview

The ideal testing cycle includes environment transformation, data preparation, traffic safety strategies, test implementation, problem analysis, and intelligent testing.

Workflow and Management

Since 2013, Alibaba has accumulated seven years of experience in full‑chain testing, evolving from large‑scale, overnight, all‑hands projects to streamlined, intelligent processes managed through strict workflow control and clear division of responsibilities.

Environment Transformation

Testing reuses the real production environment to obtain authentic results. Two main issues are addressed: business‑level concerns and data‑transfer challenges. Solutions involve both business‑level and middleware‑level transformations, supported by Alibaba’s productized one‑stop platform.

Business Transformation

Distinguish test traffic from production traffic and make it identifiable across the full stack.

Handle single‑transaction issues such as repeated order failures.

Adjust rate‑limiting mechanisms to allow real‑time configuration changes during tests.

Exclude test data from reporting and enable dynamic verification.

Middleware Transformation

Middleware passes traffic tags downstream to the database layer. Over the years, Alibaba has tackled challenges like comprehensive refactoring, code‑change costs, and version compatibility. The resulting traffic model is illustrated below:

Data Preparation

Data preparation consists of building business‑model data and test‑traffic data. Business‑model data defines which APIs are involved and their relative load ratios. Alibaba samples, filters, and masks production data to create realistic, scaled‑down datasets.

Business Model Data

The model abstracts business scenarios into executable test models, combining existing and new business flows to form a final composite model. Diagrams illustrate the process:

Baseline Test Data

Baseline data includes realistic buyer, seller, and product information. Alibaba uses shadow tables that mirror production schemas but remain isolated, ensuring test writes do not corrupt live data.

Traffic Safety Strategy

Two layers of safety are enforced:

Strict isolation of test data via shadow tables to prevent data corruption.

Filtering test traffic so it is not mistaken for attack traffic, achieved by integrating security policies with flow‑control and degradation mechanisms.

Alibaba also connects third‑party services (e.g., payment and SMS) to the testing system, enabling end‑to‑end validation.

Test Implementation

After preparation, testing proceeds with pre‑heating, login preparation, and formal execution. Formal tests include:

Peak pulse: simulate the exact traffic peak at the start of the promotion.

System ramp‑up: raise load beyond the target to discover system limits.

Rate‑limit verification: validate protection mechanisms (e.g., AHAS).

Destructive testing: conduct disaster‑recovery drills during sustained load.

External customers can configure multiple rounds of testing with varying load levels.

Problem Diagnosis and Analysis

Post‑test, Alibaba aggregates monitoring data, generates detailed PTS reports, and performs bottleneck analysis. Architecture‑level monitoring helps pinpoint issues for subsequent remediation.

Intelligent Testing

Alibaba’s full‑chain testing has evolved into a smarter system, offering features such as multi‑protocol support, capacity evaluation, automated problem detection, full‑link functional rehearsal, continuous testing, and elastic scaling during promotions.

Future Outlook

Entering its seventh year, Alibaba will continue refining its testing methodology, leveraging new technologies to better serve external customers, reduce their learning curve, and extend full‑chain testing to everyday scenarios.