Master Linux Server Monitoring: top, vmstat, pidstat, iostat, netstat & More

CPU and Memory

top

top

The first line shows the 1, 5, and 15‑minute load averages; values exceeding the number of CPU cores indicate CPU saturation. The second line lists task states: running, sleeping (interruptible/uninterruptible), stopped, and zombie processes. The third line breaks down CPU usage into user (us), system (sy), nice (ni), idle (id), iowait (wa), hardware interrupt (hi), software interrupt (si), and steal (st) percentages, each with specific meanings for performance analysis.

High user usage points to CPU‑intensive processes, system usage often reflects heavy I/O, nice indicates deliberately lowered priority, iowait signals slow I/O, and steal can reveal over‑provisioned virtual machines.

The fourth and fifth lines report physical and virtual memory. total = free + used + buff/cache. "Buffers" cache raw disk metadata, while "Cached" stores file data. "Available" memory approximates free + buffers + cache and shows how much memory is usable without swapping. Frequent swap activity suggests memory pressure.

Note that top itself consumes resources and is best for real‑time monitoring rather than long‑term data collection.

vmstat

vmstat

provides a concise snapshot of system activity. Columns include r (runnable processes), b (blocked/uninterruptible), swpd (used swap), bi/bo (blocks read/written per second), in (interrupts per second), cs (context switches per second), and others. It helps correlate CPU, memory, and I/O behavior.

pidstat

pidstat -w -t -C "<process_name>" -l

pidstat offers per‑process statistics: -r shows page faults (minor and major), -s displays stack usage, -u reports CPU usage, and -w details thread context switches (voluntary vs. involuntary). It can filter by command name with -C and show full command lines with -l, making it ideal for deep analysis of multithreaded applications.

Other CPU Tools

For per‑CPU inspection, mpstat -P ALL 1 shows load distribution across cores. Filtering top by user (e.g., top -u username) or using ps with custom columns can isolate specific processes. A loop such as

while :; do ps -eo user,pid,ni,pri,pcpu,psr,comm | grep 'process'; sleep 1; done

provides continuous monitoring.

Disk I/O

Tools like iotop display real‑time disk read/write rates per process, while lsof reveals which processes hold open files or devices, useful for diagnosing un‑unmountable partitions. iostat -xz 1 reports key metrics: avgqu-s (average queue length), await (average I/O wait time), svctm (service time), and %util (device utilization). Values >1 for avgqu-s or %util >60 % indicate potential bottlenecks.

These metrics also apply to network file systems.

Network

Network performance can be inspected with iptraf for simple throughput, or sar -n DEV 1 for detailed per‑interface statistics.

netstat

netstat -s

shows protocol‑level counters since boot. Use options like -antp to list all TCP connections or -nltp for listening sockets.

sar (network)

sar -n TCP,ETCP 1

reports active (outgoing) and passive (incoming) TCP connections, retransmissions, and input errors. sar -n UDP 1 shows UDP statistics such as no‑port packets and input errors.

tcpdump

tcpdump

captures packets for offline analysis with Wireshark. Use size‑limiting options -C and -W to rotate files, and apply filters (e.g., host, port, protocol) to reduce overhead and focus on relevant traffic.

These tools together enable comprehensive monitoring and troubleshooting of Linux servers across CPU, memory, disk, and network layers.