Mastering System Throughput: Key Metrics, Formulas, and Performance Testing Basics

System Throughput Measurement Elements

A system's throughput (capacity) is closely related to CPU consumption per request, external interfaces, I/O, etc. Higher CPU consumption per request, slower external interfaces or I/O, result in lower throughput.

Key parameters: QPS (TPS), concurrency, response time.

QPS (TPS) : number of requests/transactions per second.

Concurrency : number of requests/transactions the system handles simultaneously.

Response Time : usually the average response time.

Relationship: QPS (TPS) = Concurrency / Average Response Time .

System throughput is determined by QPS and concurrency. When either reaches its limit under load, throughput stops increasing and may decline due to overload.

Factors Determining Response Time

Similar to a project’s critical path, system response time follows a critical path composed of CPU processing, I/O, and external system responses.

System Throughput Evaluation

When designing a system, consider CPU, I/O, and external response impacts to estimate performance. Besides QPS and concurrency, daily PV is another dimension for capacity estimation.

Typical method:

Identify the system’s peak TPS and daily PV; they have a relatively stable relationship (excluding holidays or seasonal effects).

Obtain peak TPS via stress testing or experience, then calculate daily throughput using the TPS‑PV ratio.

Examples:

Taobao: TPS to PV ratio ≈ 1 : 11 × 3600 (peak TPS for 11 hours).

B2B Chinese site: ratio roughly 1 : 8 hours, varying by scenario.

If peak TPS is 100, daily throughput ≈ 100 × 11 × 3600 = 3.96 million.

Relationship among TPS, concurrency, and average response time (stable operation): TPS = Concurrency / (Response Time + Think Time).

Basic Concepts and Formulas for Software Performance Testing

1. Focus Areas of Software Performance

From a user’s perspective, the perceived performance is the time from an action (click, request) to the result being displayed—i.e., response time.

From an administrator’s perspective, key points include response time, server resource usage, application and database resource usage, scalability, maximum concurrent users, bottlenecks, hardware upgrades, and 24/7 availability.

From a developer’s perspective, considerations are architecture design, database design, code performance issues, memory usage, thread synchronization, and resource contention.

A performance test engineer must monitor all of the above.

2. Key Performance Terms

Response Time : time to respond to a request.

Network transmission time: N1+N2+N3+N4

Application server processing time: A1+A3

Database server processing time: A2

Overall response time = N1+N2+N3+N4+A1+A3+A2

Concurrent Users Calculation

System user count: total licensed users (e.g., 5000 for an OA system).

Simultaneous online users = RPS (throughput) + concurrent connections + average think time.

Average concurrent users: C = nL / T, where n = average daily active users, L = average session length, T = observation period.

Peak concurrent users: Ĉ ≈ C + 3√C (Poisson distribution).

Throughput Calculation

Throughput = number of requests processed per unit time (e.g., requests/second, pages/second, users/day).

Network‑level throughput measured in bytes/second; application‑level throughput reflects server load capacity.

When no bottleneck exists, throughput relates to virtual users: F = VU × R / T, where F is throughput, VU is virtual users, R is requests per user, T is test duration.

Performance Counters

Metrics describing server or OS performance (e.g., memory usage, CPU utilization) are essential for monitoring and bottleneck analysis.

Think Time Formula

Think Time (TS) models the interval between user actions. In throughput formula F = VU × R / T, R can be expressed as R = T / TS, leading to TS = T / R.

Typical steps to calculate think time:

Calculate concurrent users: C = nL / T, then F = R × C.

Determine average system throughput: F = VU × R / T.

Compute average requests per user: R = u × C × T / VU.

Derive think time: TS = T / R.