How to Measure and Interpret Linux Disk I/O Performance Metrics

In a previous article I discussed how Linux disk I/O works, describing the storage I/O stack composed of the file system layer, the generic block layer, and the device layer.

The generic block layer is the core of Linux disk I/O, providing a standard interface upward to file systems and applications and abstracting heterogeneous disks downward.

Linux Disk Performance Metrics

When measuring disk performance, five common metrics are used: utilization, saturation, IOPS, throughput, and response time.

Utilization : the percentage of time the disk spends processing I/O; values above 80% often indicate a bottleneck.

Saturation : how busy the disk is; 100% saturation means the disk cannot accept new I/O requests.

IOPS : number of I/O requests per second.

Throughput : amount of data transferred per second.

Response time : interval between issuing an I/O request and receiving a response.

Note: Utilization only considers whether I/O occurs, not its size, so a disk at 100% utilization may still accept new requests.

When selecting a server for an application, benchmark the disk I/O performance under random and sequential reads and writes with various I/O sizes (typically 512 B – 1 MB).

Disk I/O Observation

The most common tool for observing disk I/O is iostat, which reports metrics such as utilization, IOPS, and throughput derived from /proc/diskstats.

Example output of iostat:

# -d -x means display all disk I/O performance
$ iostat -d -x 1
Device            r/s   w/s   rkB/s   wkB/s  %util
loop0            0.00  0.00    0.00    0.00   0.00
loop1            0.00  0.00    0.00    0.00   0.00
sda              0.00  0.00    0.00    0.00   0.00
sdb              0.00  0.00    0.00    0.00   0.00

Key fields to watch: %util – disk utilization. r/s and w/s – IOPS for reads and writes. rkB/s and wkB/s – throughput. r_await and w_await – response time. iostat does not directly provide saturation; typically you compare average queue length or wait time against benchmark results (e.g., from fio) to assess saturation.

Process I/O Observation

Beyond per‑disk metrics, monitoring I/O per process is important. While iostat shows overall disk activity, tools like pidstat and iotop reveal per‑process I/O.

Example using pidstat:

$ pidstat -d 1
13:39:51   UID   PID  kB_rd/s  kB_wr/s  kB_ccwr/s  iodelay  Command
13:39:52   102  916   0.00     4.00     0.00       0       rsyslogd

pidstat

reports UID, PID, read/write kilobytes per second, cancelled write size, and I/O delay (in clock cycles).

To sort processes by I/O size, use iotop, which works like top:

$ iotop
Total DISK READ: 0.00 B/s | Total DISK WRITE: 7.85 K/s
Actual DISK READ: 0.00 B/s | Actual DISK WRITE: 0.00 B/s
  TID PRIO USER   DISK READ DISK WRITE SWAPIN  IO> COMMAND
15055 be/3 root   0.00 B/s   7.85 K/s   0.00 % 0.00 % systemd-journald

The first two lines show total disk read/write versus actual read/write, which may differ due to caching, buffering, and I/O merging. Subsequent columns display thread ID, I/O priority, per‑second read/write, swap‑in percentage, and I/O wait percentage.

Conclusion

The article introduced Linux disk I/O performance metrics—IOPS, throughput, utilization, saturation, and response time—and demonstrated how to retrieve them with iostat, as well as how to monitor per‑process I/O using pidstat and iotop. Effective analysis should consider read/write ratios, I/O types, and request sizes together.