Choosing and Implementing Performance Testing Strategies for Multi‑Business E‑Commerce Systems

Background

1. In an e‑commerce environment, every major promotion requires routine full‑link load‑testing to ensure system stability. 2. As the number of projects grows, many new or incremental services are launched without prior performance verification, creating hidden risks; therefore a bi‑monthly routine testing schedule was introduced. 3. New projects also need dedicated testing tasks to validate stability. Overall, diverse business scenarios demand low‑cost, high‑efficiency testing solutions, which this article explores.

Preparation Process Before Testing

1. Personnel

Assign dedicated QA engineers for load testing.

Allow each business team to flexibly involve their QA staff.

2. Timing of Involvement

For major promotions, start testing 15‑20 days in advance.

For routine testing, start at any time.

3. Execution Efficiency

Unattended (scheduled) testing.

Combine unattended testing with occasional onsite checks to reduce manpower.

4. Test Results

Automatically generate test reports.

Automatically push alerts to responsible owners when service exceptions occur.

Pre‑test Actions

1. Dedicated Test Users Create a set of synthetic user accounts in the system and feed them into the testing platform to avoid generating new data for each run.

2. Test Data Preparation Prepare parameterized data such as product IDs, attribute IDs, coupons, etc., using tools to batch‑pull data and improve QA efficiency.

3. Service Call Quota Request Based on the test plan and historical traffic, request the required call quota from the service governance platform to prevent throttling during testing.

4. Business Configuration Checks Verify all service configurations in the production environment before testing to avoid impacting real traffic.

Choosing the Appropriate Testing Scheme

Case Study

Normal working hours, time spent on test preparation:

Prepare test accounts: ~1 hour (when the platform does not support auto‑creation)

Identify services and interfaces: 2‑3 hours

Prepare test data: ~1 hour

Create test scenarios and debug scripts: 1‑2 hours

Total preparation: 5‑7 hours

All business teams schedule testing after midnight; a typical test requires:

RD: at least 1 person, QA: at least 1 person, test duration: 2‑4 hours

Next morning staff rest and cannot return to work immediately.

Post‑test result analysis, log collection, monitoring screenshots, and issue recording: 2‑3 hours

The case shows that a single testing task needs at least one RD and one QA, 5‑7 hours of preparation, 2‑4 hours of execution, and 2‑3 hours of result analysis.

Option 1: Onsite Testing

Traditional approach where engineers stay on‑site overnight to monitor tests. Advantages include real‑time result viewing, immediate log inspection, on‑site RD troubleshooting, and adjustable concurrency. Disadvantages are high manpower cost, potential disruption of next‑day work, and limited flexibility for scheduling.

Option 2: Scheduled (Unattended) Testing

Same preparation as onsite, but scripts run automatically at a fixed time (usually early morning). Benefits are reduced manpower, flexible scheduling, and fast frequency. Drawbacks are inability to handle special scenarios requiring configuration changes during the test and fixed concurrency settings.

Option 3: Hybrid (Onsite + Scheduled)

Combines both methods: use scheduled testing for routine or baseline checks, and onsite testing for special cases such as new projects, refactoring, bottleneck verification, or flash‑sale scenarios that need manual intervention.

Option 4: Dedicated Test Environment

Rejected due to high cost and low ROI: difficulty reproducing production conditions, complex architecture setup, and the short duration of most tests makes a separate environment uneconomical.

Final Practice Results

Option 4 was discarded. The team now employs a mix of onsite and scheduled testing based on business needs, achieving efficient, regular load‑testing for the 618 promotion while keeping costs low.

Post‑test Summary

1. Daily testing is possible only with robust platform support (automatic circuit‑break, timeout handling, fault tolerance).

2. Test participants must have performance analysis skills to provide optimization suggestions.

3. Efficiency gains come from learning from past pitfalls and establishing streamlined processes.

4. Full‑link testing is essential before major promotions, but routine load‑testing is the key to long‑term stability.

Next Steps

1. Separate production traffic from test traffic using internal routing to enable testing during regular work hours.

2. Automate aggregation of test results, including interface metrics, call‑chain analysis, overall traffic, exceptions, and latency.