Master Linux Server Performance: Essential Tools & Metrics Explained

1. CPU and Memory

A Linux server continuously reports various parameters that are crucial for operations staff and developers when diagnosing abnormal program behavior.

1.1 top

Running top shows load averages (1, 5, 15 minutes) and task states. When the load exceeds the number of CPU cores, the CPU is saturated.

(us) user : Time spent in user space with low nice values.

(sy) system : Time spent in kernel space.

(ni) nice : Time spent in user space with high nice values.

(id) idle : Idle time.

(wa) iowait : Time waiting for I/O.

(hi) irq : Time handling hardware interrupts.

(si) softirq : Time handling software interrupts.

(st) steal : Time stolen by the hypervisor in virtualized environments.

High percentages guide troubleshooting: excessive us points to CPU‑bound processes; high sy may indicate heavy I/O; high wa suggests slow storage; high st can reveal over‑committed VMs.

1.2 vmstat

vmstat

provides a compact view of processes, memory, paging, block I/O, and CPU activity. Columns include:

r : runnable processes.

b : uninterruptible sleep processes.

swpd : used virtual memory.

bi/bo : blocks received/sent to disk per second.

in : interrupts per second.

cs : context switches per second.

1.3 pidstat

pidstat -w -t -C "ailaw" -l

offers per‑process statistics, including page faults, stack usage, CPU usage, and context switches.

minflt/s : minor page faults (fast, page already in memory).

majflt/s : major page faults (require loading from disk, costly).

StkSize / StkRef : allocated vs. used stack space.

-u : CPU usage breakdown similar to top.

-w : thread context switches (cswch/s and nvcswch/s).

For quick per‑process monitoring, ps -eo user,pid,ni,pri,pcpu,psr,comm | grep 'ailawd' can be wrapped in a loop:

while :; do ps -eo user,pid,ni,pri,pcpu,psr,comm | grep 'ailawd'; sleep 1; done

1.4 Other CPU tools

On SMP systems, mpstat -P ALL 1 shows per‑core utilization and load balance.

2. Disk I/O

Tools such as iotop and lsof reveal real‑time disk read/write rates and file descriptor usage.

2.1 iostat

iostat -xz 1

reports key metrics:

avgqu-s : average queue length; >1 indicates device saturation.

await : average I/O wait time (ms).

svctm : average service time (ms).

%util : percentage of time the device is busy; >60% may degrade performance.

Even with high I/O latency, kernel asynchronous I/O and caching can mask impact on applications.

3. Network

Network health is critical; tools like netstat, sar, and tcpdump provide visibility.

3.1 netstat

Use netstat -s for protocol statistics and netstat -antp to list all TCP connections.

3.2 sar

sar -n TCP,ETCP 1

and sar -n UDP 1 report per‑second TCP/UDP activity, including active/passive connections, retransmissions, and error rates.

3.3 tcpdump

tcpdump

captures packets for offline analysis with wireshark. It supports filtering by interface, host, port, and protocol, and can limit file size with -C / -W to avoid overwhelming the system.