Master Linux System Monitoring: top, free, vmstat, iostat, and More

1.1 top

1.1.1 Command description

The top command monitors system runtime status in real time and can sort by CPU, memory, and execution time.

1.1.2 Parameter description

Command line start parameters:

Usage:

top -hv | -bcisSHM -d delay -n iterations [-u user | -U user] -p pid [,pid ...]

-b: batch mode, useful for piping output to other programs or files.

-c: display command line of running tasks.

-d: set delay time.

-h: help.

-H: show threads.

-i: show idle processes.

-n: number of iterations, usually used with -b.

-u: monitor processes of a specific user.

-U: same as -u.

-p: monitor specific process IDs (comma‑separated).

-s: safe mode.

-S: cumulative time mode.

-v: display top version and exit.

-M: auto‑display memory units (k/M/G).

1. Global commands

Enter/Space: refresh display.

?, h: help.

=: remove all task display limits.

A: toggle alternate display mode.

B: toggle bold display.

d, s: change refresh interval.

G: select other window/field group.

I: toggle Irix/Solaris mode.

u, U: monitor processes of a specific user.

k: kill process.

q: quit top.

r: reset nice value of a process.

W: save current settings.

Z: change color scheme.

2. Summary area commands

l: toggle load average and uptime display.

m: toggle memory and swap usage display.

t: toggle task and CPU state display.

1: display CPU state aggregated or per‑CPU.

3. Task area commands

Appearance

b: bold/reverse highlight rows/columns.

x: highlight sorted column.

y: highlight running tasks.

z: color/monochrome display.

Content

c: show command line or process name.

f, o: add/remove and reorder fields.

H: show threads.

S: cumulative time mode.

u: monitor specific user.

Task quantity

i: show idle processes.

n or #: set maximum number of tasks displayed.

Task sorting (shift+f)

M: sort by memory usage.

N: sort by PID.

P: sort by CPU usage.

T: sort by time+.

<: sort by field left of current.

>: sort by field right of current.

F or O: choose sort field.

R: reverse sort.

1.1.3 Result description

1.2 free

1.2.1 Command description

The free command is the most common tool for monitoring system memory.

1.2.2 Parameter description

-m: display memory in megabytes (default is kilobytes).

-b: display memory in bytes.

-s: monitor memory usage over a specified interval.

1.2.3 Result description

total: total physical memory.

Used: amount used.

Free: amount free.

shared: total memory shared among processes.

buffers/cached: size of disk cache.

1.3 vmstat

1.3.1 Command description

vmstat monitors process status, memory, virtual memory, disk I/O, context switches, and CPU information.

1.3.2 Parameter description

Usage: vmstat [-a] [-n] [-S unit] [delay [count]] -a: show active and inactive memory.

-m: show slabinfo.

-n: display field names only once.

-s: show memory‑related statistics and system activity counts.

delay: refresh interval (if omitted only one line is shown).

count: number of refreshes (infinite if omitted and delay is set).

-d: display disk statistics.

-S: display using specified unit (k, K, m, M).

-V: show vmstat version.

-p: display statistics for a specific disk partition.

-D: display overall disk information.

1.3.3 Result description

Procs

R: number of processes waiting for CPU time.

B: number of processes waiting for resources.

Memory

Swap: virtual memory size swapped.

Free: idle physical memory.

Buffers: buffer size.

Cache: cache size.

1.4 iostat

1.4.1 Command description

iostat monitors disk I/O operations and also provides CPU usage statistics.

1.4.2 Parameter description

Usage:

iostat [ -c | -d ] [ -k | -m ] [ -t ] [ -V ] [ -x ] [ device [...] | ALL ] [ -p [ device | ALL ] ] [ interval [ count ] ]

-c: display only CPU statistics (mutually exclusive with -d).

-d: display only disk statistics.

-k: display per‑second request counts in kilobytes (default is blocks).

-p device|ALL: display statistics for a specific device or all devices (mutually exclusive with -x).

-t: print timestamp with each report.

-V: show version and help.

-x device: display extended statistics for the specified device.

interval: time between reports.

count: number of reports.

1.4.3 Result description

Simple iostat usage example:

Disk monitoring fields:

rrqm/s: merged reads per second.

wrqm/s: merged writes per second.

r/s: reads completed per second.

w/s: writes completed per second.

rsec/s: sectors read per second.

wsec/s: sectors written per second.

rkB/s: kilobytes read per second.

wkB/s: kilobytes written per second.

avgrq‑sz: average request size.

avgqu‑sz: average queue length.

await: average wait time (ms).

svctm: average service time (ms).

%util: percentage of time the device was busy.

CPU monitoring fields:

%usr: user‑mode CPU percentage.

%nice: nice‑mode CPU percentage.

%system: kernel‑mode CPU percentage.

%iowait: I/O wait percentage.

%steal: stolen CPU time in virtualized environments.

%idle: idle CPU percentage.

1.5 mpstat

1.5.1 Command description

mpstat reports CPU statistics; on multi‑core systems it can show average and per‑CPU information.

1.5.2 Parameter description

Usage: mpstat [-P {|ALL}] [interval [count]] Parameters:

-P {|ALL}: select which CPU(s) to monitor (0‑N‑1 or ALL).

interval: time between samples.

count: number of samples (used with interval).

Without parameters mpstat shows averages since boot; with interval it shows per‑interval data.

1.5.3 Result description

Fields:

user: % CPU time spent in user mode.

nice: % CPU time spent in low‑priority user mode.

system: % CPU time spent in kernel mode.

iowait: % CPU time waiting for I/O.

irq: % CPU time handling hardware interrupts.

soft: % CPU time handling software interrupts.

idle: % CPU time idle (excluding iowait).

intr/s: interrupts per second.

1.6 sar

1.6.1 Command description

sar collects and reports CPU, memory, I/O, network, and other system activity statistics.

1.6.2 Parameter description

Typical usage: sar [options] [-A] [-o file] interval [count] -A: all reports.

-u: CPU utilization.

-v: process, inode, file, and lock table status.

-p: per‑CPU usage.

-d: disk usage.

-r: memory usage.

-n: network statistics (DEV, EDEV, SOCK, FULL).

-q: run queue size.

-B: memory paging.

-R: process activity.

-g: serial I/O.

-b: buffer usage.

-a: file I/O.

-c: system calls.

-y: terminal device activity.

-W: swap activity.

1.6.3 Result description

CPU resource monitoring example:

Key fields:

%user: CPU time spent in user space.

%nice: CPU time spent on nice processes.

%system: CPU time spent in kernel space.

%iowait: time waiting for I/O.

%steal: time stolen by hypervisor.

%idle: idle CPU time.

High %iowait indicates I/O bottleneck; low %idle with slow response may mean memory pressure; %idle consistently below 1 % suggests CPU is the limiting resource.

Memory and swap monitoring fields (kbmemfree, kbmemused, %memused, kbbuffers, kbcached, kbcommit, %commit) are explained, as well as paging statistics (pgpgin/s, pgpgout/s, fault/s, majflt/s, pgfree/s, pgscank/s, pgscand/s, pgsteal/s, %vmeff).

Disk I/O and transfer rates (tps, rtps, wtps, bread/s, bwrtn/s, avgrq‑sz, avgqu‑sz, await, svctm, %util) are described, with interpretation guidelines.

Process queue length and load averages (runq‑sz, plist‑sz, ldavg‑1, ldavg‑5, ldavg‑15) are shown.

Swap activity (pswpin/s, pswpout/s) is also covered.

1.7 netstat

1.7.1 Command description

netstat displays network connections, listening ports, routing tables, and interface statistics.

1.7.2 Parameter description

-a: show all connections (including listening).

-t: show TCP.

-u: show UDP.

-n: show numeric addresses.

-l: show only listening services.

-p: show program name/PID.

-r: show routing table.

-e: show extended information (e.g., UID).

-s: protocol statistics.

-c: continuous display at intervals.

-v: verbose.

-I: show interface information.

1.7.3 Result description

Iface: interface name.

MTU: maximum transmission unit.

RX‑OK/TX‑OK: packets received/sent successfully.

RX‑ERR/TX‑ERR: receive/transmit errors.

RX‑DRP/TX‑DRP: dropped packets.

RX‑OVR/TX‑OVR: overruns.

Flg: interface flags (B broadcast, L loopback, M multicast, N no‑track, O ARP off, P point‑to‑point, R running, U up, etc.).

Connection list columns (Recv‑Q, Send‑Q, Local Address, Foreign Address, State) are explained; states include LISTEN, ESTABLISHED, TIME_WAIT.

1.8 uptime

1.8.1 Command description

uptime reports the current time, how long the system has been running, number of logged‑in users, and load averages for the past 1, 5, and 15 minutes.

1.8.2 Parameter description

-V: display version.

1.8.3 Result description

If load average stays above the number of CPU cores, the system is likely overloaded.

1.9 ps

1.9.1 Command description

ps lists running processes, their status, resource usage, and other attributes.

1.9.2 Parameter description

Common options include:

-A / -e: all processes.

-a: all processes with a terminal.

-d: all processes except session leaders.

-x: processes without a controlling terminal.

-p pid: specific PID.

-u user: processes owned by user.

-g gid: processes in group.

-f: full format.

-l: long format.

-j: job format.

-o format: custom output.

-m: threads.

-H: hierarchical view.

-h: omit header.

1.9.3 Result description

Key columns: USER, UID, PID, PPID, SID, %CPU, %MEM, VSZ, RSS, TTY, STAT (R, S, I, Z, D, T, etc.), START, TIME, COMMAND, NI, PRI, WCHAN, FLAGS.

1.10 watch

1.10.1 Command description

watch runs a command periodically and highlights changes.

1.10.2 Parameter description

-d: highlight differences.

-n seconds: interval between executions.

1.10.3 Result description

Example: watch -d -n 1 netstat -ant monitors network connections every second.

1.11 strace

1.11.1 Command description

strace traces system calls and signals of a process, showing arguments, return values, and execution time.

1.11.2 Parameter description

-p pid: trace a specific process.

-f: follow forked children.

-F: follow vfork (unless -f also present).

-o file: write output to file.

-ff: separate output per PID.

-r: show relative time per call.

-t / -tt / -ttt: timestamp each line (microseconds or relative).

-v: verbose (show all syscalls).

-s N: max string length (default 32).

-c: summarize time and count per syscall.

-e expr: filter output.

-d: debug output.

-i: show syscall entry pointer.

-q: suppress detach messages.

-T: show time spent in each call.

-V: version.

-x / -xx: hex dump of non‑printable strings.

1.11.3 Result description

Example: strace -ff -F -o ls.log ls -l traces the execution of ls -l.

If a function fails, the return value is typically –1.

1.12 lsof

1.12.1 Command description

lsof lists open files and the processes that opened them; in Linux everything is a file.

1.12.2 Parameter description

-a: list processes with open files.

-c name: list files opened by a specific command.

-g: list by GID.

-d fd: list processes using a specific file descriptor.

+d dir: list files opened under a directory.

+D dir: recursively list files under a directory.

-n: list NFS files.

-i criteria: list processes matching network criteria.

-p pid: list files opened by a specific PID.

-u user: list files opened by a user.

-U: list only socket files.

-h: help.

-v: version.

1.12.3 Result description

Example: list all socket files owned by root.

COMMAND: process name.

PID: process ID.

USER: owner.

FD: file descriptor (cwd, txt, etc.).

TYPE: file type (DIR, REG, etc.).

DEVICE: device name.

SIZE: file size.

NODE: inode.

NAME: exact file name.