Master Linux I/O Monitoring: iostat, iotop, and lsof Explained

iostat

The iostat tool monitors disk operation activity and reports both disk statistics and CPU usage. It requires the sysstat package.

yum -y install sysstat
iostat
Linux 5.14.0-162.6.1.el9_1.x86_64 (localhost.localdomain) 12/02/22 _x86_64_ (2 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           1.75    0.02    5.84    0.60    0.00   91.79

Device             tps    kB_read/s    kB_wrtn/s    kB_dscd/s    kB_read    kB_wrtn    kB_dscd
 dm-0             73.70      2078.84       358.65         0.00     162919      28107          0
 dm-1              1.26        29.96         0.00         0.00      2348          0          0
 nvme0n1         104.45      2809.71       385.01         0.00    220197      30173          0
 sr0               1.03        40.12         0.00         0.00      3144          0          0

Key fields:

avg-cpu : %user (user‑space CPU), %nice (nice‑adjusted CPU), %system (kernel CPU), %iowait (CPU waiting for I/O), %idle (idle CPU).

Device : tps (transactions per second), kB_read/s, kB_wrtn/s, kB_dscd/s, plus cumulative totals.

Common options: -c: display only CPU statistics (mutually exclusive with -d). -d: display only disk statistics. -k: show values in kilobytes per second (default is blocks). -p <device|ALL>: limit output to a specific device or all devices. -x: display extended statistics. sec: interval in seconds between reports.

iotop

iotop

provides a top‑like interface for monitoring real‑time disk I/O per process. It is not installed by default and requires the iotop package.

yum -y install iotop
iotop
Total DISK READ :       0.00 B/s | Total DISK WRITE :       0.00 B/s
Actual DISK READ:       0.00 B/s | Actual DISK WRITE:       0.00 B/s
    TID  PRIO  USER     DISK READ DISK WRITE  COMMAND
      1 be/4 root        0.00 B/s    0.00 B/s systemd --switched-root --system --deserialize 28
      2 be/4 root        0.00 B/s    0.00 B/s [kthreadd]
      3 be/4 root        0.00 B/s    0.00 B/s [rcu_gp]
      ... (additional rows omitted for brevity) ...

The display shows each thread’s ID, priority, owning user, read/write bandwidth, and the command line, allowing you to identify which processes are generating I/O load.

Frequently used options: -b: batch mode, suitable for logging to a file. -n NUM: refresh NUM times and then exit (useful for non‑interactive scripts). -d SEC: set the refresh interval to SEC seconds. -p PID: monitor only the specified process ID. -u USER: monitor processes owned by USER.

lsof

lsof

(list open files) enumerates all files opened by processes. It requires root privileges for full information and the lsof package.

yum -y install lsof
lsof
COMMAND    PID TID TASKCMD     USER   FD      TYPE   DEVICE SIZE/OFF       NODE NAME
systemd      1                 root  cwd       DIR    253,0      235        128 /
systemd      1                 root  rtd       DIR    253,0      235        128 /
systemd      1                 root  txt       REG    253,0  1945080   67508326 /usr/lib/systemd/systemd
systemd      1                 root  mem       REG    253,0    45416   67348788 /usr/lib64/libffi.so.8.1.0
... (additional rows omitted for brevity) ...

Important columns:

PID : Process identifier.

USER : Owner of the process.

FD : File descriptor (e.g., cwd, txt, mem).

TYPE : File type (DIR, REG, etc.).

DEVICE : Device identifier.

SIZE/OFF : Size or offset.

NODE : Inode number.

NAME : Full path of the opened file.

Common usage examples: lsof -c abc – show files opened by processes whose command name matches abc. lsof -p 1234 – list files opened by process ID 1234. lsof -g gid – display files opened by processes belonging to group ID gid. lsof +d /DIR/ – list all files opened under directory /DIR/.