Essential System Performance Metrics and Linux Server Monitoring Guide

Performance testing metrics are standards for evaluating system performance. Common metrics include response time, concurrent users, click‑through rate, throughput, TPS/QPS, and PV/UV. Typical Linux server performance indicators are CPU usage, memory usage, disk I/O, and system load average.

1 System Performance Testing Metrics

1.1 Response Time

Response time is the total time from issuing a request to receiving feedback, covering client processing, network transmission, and database processing. For example, the time from clicking a page to full load, or the time to complete an add, delete, update, or query transaction.

A request traverses multiple network nodes (B1, B2, B3) before reaching the server. The client processing time is A and the server response time is C, so the full path is A → B1 → B2 → B3 → C.

Thus, response time = A + B1 + B2 + B3 + C.

1.2 Concurrency

Concurrency refers to multiple users performing the same transaction or operation at the same period. It is divided into absolute concurrency (simultaneous requests at a single instant) and relative concurrency (requests within a time interval).

For example, a zoo that can hold 10,000 visitors simultaneously actually describes relative concurrency, because visitors follow different paths and the entrance can only accommodate about 200 people at the same moment, which is absolute concurrency.

In system performance testing, concurrency usually means relative concurrency for modules, while interface concurrency tends to refer to absolute concurrency. Two common metrics are concurrent user count and concurrent request count.

1.3 Clicks / Click‑Through Rate

Clicks (clicks) count the number of times a page element is clicked. Click‑through rate (CTR) is the ratio of clicks to impressions (clicks/views), indicating how attractive a piece of content or advertisement is.

In performance testing, hit rate often means the number of HTTP requests per second; a higher hit rate puts more pressure on the server.

1.4 Throughput / Throughput Rate

Throughput is the total number of client requests processed by the system, which can refer to total network packets or total data exchanged between client and server.

Throughput rate is the amount of work done per unit time. It can be measured from three perspectives:

Business: requests per second or pages per second.

Network: bytes transferred per second.

System: the load the server can sustain per second.

Throughput rate is closely related to CPU, memory, I/O, and network bandwidth consumption.

1.5 TPS / QPS

TPS (Transactions Per Second) counts how many business transactions are completed per second. A transaction may consist of multiple queries or a single request.

QPS (Queries Per Second) counts how many queries or server accesses occur per second.

The difference is that one TPS may correspond to multiple QPS when a transaction involves several queries.

1.6 PV / UV

PV (Page View) records each page access or click, counting every visit even from the same user.

UV (Unique Visitor) counts distinct visitors within a time window; repeated accesses from the same client are counted only once.

PV and UV can be aggregated for the whole day, per hour, or for peak periods during large shopping events.

2 Linux Server Performance Indicators

2.1 CPU Usage

CPU usage is the percentage of CPU time spent on non‑idle activities. It can be calculated as 1 – (idle time / total CPU time).

#top (updates every 3 seconds)

Key fields:

us (user): time spent in user mode.

sy (system): time spent in kernel mode.

ni (nice): time spent on low‑priority user processes.

id (idle): idle time proportion.

wa (iowait): time waiting for I/O.

hi (hard interrupt): time handling hardware interrupts.

si (soft interrupt): time handling software interrupts.

st (steal): time stolen by other virtual machines.

In performance testing, overall CPU usage can be approximated by (1 – id). High us suggests user‑code bottlenecks, high sy points to kernel overhead, and high wa indicates I/O contention. Generally, if us + sy ≤ 70 %, the system is considered healthy.

2.2 Memory Usage

Linux maximizes memory utilization by using free memory for caching. Cached memory stores frequently read files, while buffers cache block‑device metadata.

Formulas:

MemUsed = MemTotal – MemFree (includes buffers and cached).

-/+ Used = MemTotal – MemFree – MemBuffers – MemCached.

MemUsed% = (MemUsed / MemTotal) × 100 %.

-/+ Used% = (‑/+ Used / MemTotal) × 100 %.

When memory usage exceeds 70 %, the system may be under memory pressure.

#top

2.3 System Load Average

Load average shows the average number of processes in runnable or uninterruptible states over the past 1, 5, and 15 minutes. Commands #uptime, #top, and #w display these values.

#uptime

#top

#w

If the 1‑minute load exceeds the 5‑ or 15‑minute values, load is increasing; if it is lower, load is decreasing. When load exceeds the number of CPU cores, the system may encounter performance bottlenecks.

2.4 Disk I/O

Disk I/O is another crucial performance metric for Linux servers.

#iostat -x -k 2 3 (samples every 2 seconds, three times)

Key fields:

rrqm/s: read requests merged per second.

wrqm/s: write requests merged per second.

r/s: reads completed per second.

w/s: writes completed per second.

rkB/s: kilobytes read per second.

wkB/s: kilobytes written per second.

avgrq‑sz: average size of I/O requests.

avgqu‑sz: average queue length.

await: average wait time (ms) per I/O request.

svctm: average service time (ms) per I/O request.

%util: percentage of time the device was busy.

In performance testing, focus on iowait% and %util. An iowait% above 50 % or significantly higher than %sys, %user, and %idle indicates possible I/O problems. Generally, %util ≤ 70 % means disk I/O is healthy.