Three Core Dimensions of Performance Testing: Time Behavior, Resource Utilization, and Capacity

1. Core dimensions and key metrics – Performance testing is organized around three pillars: (a) Time behavior (response time, processing time, latency, turnaround time) with metrics such as average response time (ART), P95/P99, TPS, QPS; (b) Resource utilization (CPU, memory, disk I/O, network bandwidth, connection counts) with metrics like CPU usage, memory RSS, GC pause, thread‑pool usage; (c) Capacity (maximum concurrent users, data volume, storage limits, scalability, bottleneck identification) with metrics such as max concurrent users, table row limits, file counts.

2. 3‑D relationship model – Time behavior maps to user‑experience speed, resource utilization to cost efficiency, and capacity to business‑support limits; the three dimensions influence each other (e.g., faster response may increase CPU usage).

3. Case study: large‑scale e‑commerce flash‑sale system – Scenario: 10 million users in 10 minutes buying 10 000 iPhones (price 5999 CNY). Architecture includes micro‑services, Redis cluster, message queue, CDN, sharded databases. Test goals: 99 % of requests < 500 ms, CPU < 70 %, memory < 80 %, support 500 k concurrent users, process 50 k orders per second.

4. Test design

Time‑behavior tests : simulate traffic spikes, network variance, end‑to‑end latency. Example test cases – TC‑T1 (product page load, 100 k users, ART < 200 ms, P95 < 300 ms, TPS > 50 k/s); TC‑T2 (order flow, 50 k users, response < 500 ms, success > 99.9 %, TPS > 30 k/s); TC‑T3 (payment, 10 k users, avg < 1 s, timeout < 0.1 %).

Resource‑utilization tests : high‑concurrency resource consumption, component bottlenecks, GC impact. Example cases – TC‑R1 (core services monitoring CPU < 70 % peak, memory < 80 % peak, GC < 100 ms); TC‑R2 (Redis cluster 6→12 nodes, CPU < 60 %, memory < 70 %, bandwidth < 1 Gbps, connections < 50 k/node); TC‑R3 (MySQL sharded, connections < 1 k/instance, I/O < 10 ms, slow‑query < 0.01 %).

Capacity tests : step‑wise load increase until failure, verify scaling. Example cases – TC‑C1 (capacity limit, start 100 k users, increase by 50 k every 10 min, target > 500 k concurrent, graceful degradation at 600 k); TC‑C2 (data volume, 10 M product rows, 100 M user rows, 1 B order rows, query performance degradation < 20 % when data doubles); TC‑C3 (elastic scaling, add service instances from 10→20, Redis nodes 6→12, expect scaling latency < 5 min and linear or super‑linear throughput).

5. Test environment – 200 load‑generator machines, distributed deployment, network emulation for regional latency, realistic traffic mix (70 % mobile, 30 % PC, 80 % browse, 15 % add‑to‑cart, 5 % purchase), 1 B historical orders, real‑time monitoring with Prometheus + Grafana, APM tools, JVM profilers, OS metrics (top, vmstat, iostat), chaos‑engineering (ChaosBlade) for fault injection.

6. Result analysis & optimization

Time‑behavior issue : response spiked > 2 s at flash‑sale start; root cause – DB connection pool exhaustion. Optimizations – Redis cache warm‑up, two‑level cache, request merging.

Resource‑utilization issue : order service memory ≈ 90 % with frequent Full GC; root cause – JSON serialization overhead. Optimizations – switch to Protobuf, tune JVM GC, object pooling.

Capacity issue : at 400 k users DB connections became bottleneck; root cause – long‑held connections. Optimizations – read‑write splitting, connection‑pool timeout, async processing.

After tuning, key improvements were observed: P95 response time reduced from 800 ms to 350 ms (‑56 %); order TPS increased from 20 k/s to 45 k/s (+125 %); CPU peak from 85 % to 65 % (‑23 %); memory peak from 90 % to 75 % (‑17 %); max concurrent users raised from 400 k to 700 k (+75 %).

7. Testing strategy matrix – Combines benchmark, load, stress, endurance, and spike tests, each focusing on specific metrics (single‑user latency, per‑request resource cost, single‑machine capacity, SLA‑compliant load, resource bottleneck identification, system limits, long‑run stability, and elastic scaling).

8. Key success factors – Production‑like environment, realistic traffic simulation, full‑stack tracing, progressive testing from baseline to stress, continuous performance testing in CI/CD, mapping technical metrics to business KPIs.

Conclusion – Balancing time behavior, resource utilization, and capacity yields an optimal performance state that satisfies user experience, operational cost, and business growth. The flash‑sale case demonstrates how systematic testing and data‑driven optimization achieve that balance.