Master Linux I/O Monitoring: iostat, iotop, sar, dstat, nmon, Zabbix & Cacti

1. System‑provided I/O tools

1.1 iostat

The iostat command from the systat package reports I/O statistics in kB/s. Use iostat -x -k 2 10000 to sample every 2 seconds for 10 000 intervals.

Key fields:

rrqm/s – read requests merged per second

wrqm/s – write requests merged per second

rsec/s – sectors read per second

wsec/s – sectors written per second

rKB/s, wKB/s – read/write kilobytes per second

avgrq‑sz – average request size

avgqu‑sz – average queue length (shorter is better)

await – average I/O response time (µs/ms); < 5 ms is ideal, > 10 ms indicates a problem

svctm – average service time; if await ≈ svctm, queueing is minimal

%util – proportion of time the device was busy (≈100 % means near full load)

1.2 iotop

Install with yum install iotop. Run iotop -d 1 -o to display only processes performing I/O, refreshed every second.

Common options:

-o – show only processes with I/O activity

-b – batch mode (no interactive UI, suitable for logging)

-n NUM – display NUM updates

-d SEC – refresh interval in seconds

-p PID – monitor a specific PID

-u USER – monitor processes of a specific user

1.3 sar

Typical usage: sar -p -d 2 1000 or sar -b 2 1000. Important metrics:

tps – total I/O requests per second (reads + writes, after merge)

rtps – read requests per second

wtps – write requests per second

bread – bytes read per second

bwrtn – bytes written per second

1.4 dstat dstat combines the functionality of vmstat, iostat, netstat, nfsstat and ifstat. It provides a colorful, real‑time view of system resources. Example: dstat 3 collects data every 3 seconds and refreshes each second.

Common options:

-c – CPU usage

-C – per‑CPU statistics (e.g., -C 0,1)

-d – disk read/write size

-D DEV,total – include specific device and totals

-n – network statistics

-N IFACE,total – specific network interface

-l – system load

-m – memory usage

-g – page usage

-p – process statistics

-s – swap usage

-r – I/O request statistics

-y – overall system status

--ipc – IPC queues and signals

--socket – TCP/UDP socket status

-a – default (equivalent to -cdngy)

-v – verbose (equivalent to -pmgdsc -D total)

--output FILE – write CSV output for later analysis

2. Dedicated monitoring solutions

2.1 nmon

Installation on Linux:

[root@host]# wget http://sourceforge.net/projects/nmon/files/download/nmon_x86_12a.zip/download
[root@host]# unzip download
[root@host]# yum install ld-linux.so.2 libncurses.so.5
[root@host]# chmod +x nmon_x86_rhel52
[root@host]# mv nmon_x86_rhel52 /usr/local/bin/nmon
# Collect data every 10 s for 60 samples (10 min)
[root@host]# nmon -s10 -c60 -f -m ./
# Verify background process
[root@host]# ps -ef | grep -i nmon

Analysis can be performed with the nmon_analyser utility or by opening the generated CSV in Excel.

2.2 Zabbix

Zabbix provides ready‑made IOPS monitoring templates that can be imported directly, producing graphs of disk activity.

2.3 Cacti

Cacti is a mature graphing solution. After installing the IOPS template (available at http://forums.cacti.net/about8777.html) follow the standard deployment steps and import the provided XML files.

All the tools above can monitor CPU, memory, disk space, and network traffic, making them versatile choices for comprehensive system performance analysis.