How to Design Effective Performance Test Scenarios with JMeter

Background

According to a 2008 Aberdeen Group study, a one‑second page‑load delay can reduce page views by 11% and customer satisfaction by 16%, potentially costing a site that earns ¥100,000 daily up to ¥250,000 in annual sales. Compuware’s analysis of 150 sites and 1.5 million page views showed that response times increasing from 2 seconds to 10 seconds raise abandonment rates by 38%, underscoring the business importance of performance testing.

Performance testing typically covers four scenarios—baseline, capacity, stability, and exception—to meet diverse performance requirements.

Implementation Overview

The testing process follows a disciplined workflow, likened to the ancient saying “without rules, one cannot form a square.” After establishing the workflow, testers can proceed without ad‑hoc actions.

Key steps include preparing the environment (preferably mirroring production hardware, software, and container configurations), preparing realistic data volumes, and ensuring parameterized data reflects production distributions.

Baseline Scenario

The baseline scenario focuses on single‑interface load testing.

Environment preparation: online testing for internet companies; isolated test environments for non‑internet or legacy sectors.

Data preparation: replicate production data volumes; ensure database snapshots are available for troubleshooting.

Parameter preparation: generate sufficient varied data to simulate realistic user behavior; avoid using only a few records.

Parameter sources can be existing production databases or synthetic data generated by the load tool. Valid parameter data must satisfy production data distribution and volume requirements.

Baseline testing ends when resource utilization (CPU, memory, I/O) reaches ~90% or the system is fully saturated, ensuring TPS and response times are realistic.

Capacity Scenario

The capacity scenario combines multiple interfaces proportionally to answer “what is the maximum online capacity?” Production logs (or ELK/ELFK pipelines) are used to derive request ratios. If logs are unavailable, Nginx logs or custom Python/Shell scripts can extract the needed data.

After calculating each interface’s request proportion, designers create a composite test that respects these ratios, using JMeter’s Throughput Timer and Throughput Controller to control TPS.

Stability Scenario

The stability scenario evaluates long‑duration performance to detect cumulative effects. Test duration is calculated based on business volume and maximum TPS, e.g., 60 million transactions ÷ 1,000 TPS ÷ 3,600 seconds ≈ 16 hours.

Exception Scenario

Exception testing targets architectural and capacity‑induced failures. Common techniques include host power‑off/reboot, network interface down, packet loss simulation, and application kill/stop. Designing these scenarios benefits from a failure‑mode‑effects‑analysis (FMEA) approach to ensure comprehensive coverage.

Overall, the four‑scenario framework helps align performance testing with real‑world traffic patterns, ensuring test results are meaningful and actionable.