Why IO Aggregation and RAID Penalties Matter: Deep Dive into Storage Performance

Introduction

The article builds on a previous discussion of storage performance evaluation concepts and focuses on practical analysis of end‑to‑end bottlenecks, including host ports, storage subsystems, and backend disks. It lists common difficulties encountered during performance assessment and offers guidance for realistic configuration based on real‑world workloads.

IO Aggregation and RAID Write Penalty

When IO is aggregated to full‑stripe size, pre‑read is unnecessary and RAID write penalty is avoided. For RAID5‑5 (4D+1P), a single data write normally triggers two pre‑reads and one parity write, expanding one IO into four. In full‑stripe writes, four data IOs are written together with a single parity write, expanding four IOs into five, dramatically improving efficiency.

Storage IO Merge Capability

IO merge ability varies by storage vendor and depends on two factors: (1) the host‑side IO model (sequential vs. random, block size, HBA distribution, etc.) and (2) the storage‑side merge mechanisms (cache, block devices, disks). Sequential small IOs can often be merged into full‑stripe writes, while random database workloads achieve only partial merge due to algorithmic and memory constraints.

Typical Business Models and Their IO Characteristics

OLTP : Small, random 8 KB IOs, read/write ratio ≈3:2, latency 10‑20 ms.

OLAP : Mostly large sequential reads (≈512 KB), >90 % reads, occasional mixed write on temporary LUNs.

VDI : Mixed storm phases – start‑up (read‑intensive), login (write‑intensive), steady state (≈2:8 read/write), IO size 512 B‑16 KB.

SPC‑1 : Industry‑standard random IO model, read/write ≈4:6, IO size ≈4 KB, mixed sequential/random ratio ≈3:7.

FC Bandwidth Calculation

For an 8 Gbps FC link, the theoretical one‑way bandwidth is calculated as:

LinkClock * EncodingEfficiency * FCProtocolEfficiency / 8 / 1024 / 1024

Using 8.5 GHz clock, 8b/10b encoding, and 97.15 % protocol efficiency yields 787.5 MB/s one‑way. Real‑world bandwidth is lower due to protocol overhead and module scheduling.

Impact of Parity Checks on Effective Bandwidth

In sequential writes that reach full‑stripe size, RAID5‑5 (4D+1P) adds one parity write for every four data writes, consuming disk bandwidth without providing usable capacity. For reads, full‑stripe reads need only data disks, not parity disks, further improving effective bandwidth.

Read/Write Ratio and Performance

The read/write ratio directly influences cache strategy, RAID level choice, and LUN configuration. Write‑heavy workloads consume more resources due to longer IO paths, write‑hole mechanisms, and RAID write penalties. Example calculations show that a 2:8 write‑heavy mix can require five times more disk IO than an 8:2 read‑heavy mix under RAID6.

RAID Level Performance and Capacity Trade‑offs

RAID10, RAID5, and RAID6 exhibit different effective performances after accounting for write penalties. For identical disk counts, RAID6 provides the highest reliability but the lowest usable capacity and performance under write‑intensive scenarios. Capacity loss is proportional to the number of parity disks.

Sequential vs. Random IO

Sequential IO benefits from lower seek time on mechanical disks and higher cache hit rates. Random IO suffers from higher latency and lower IOPS, especially for small IO sizes. Storage systems use prefetch and merge algorithms to convert sequential small IOs into larger ones, improving both read and write efficiency.

IO Size Impact

IOs ≤16 KB are considered small; ≥32 KB are large. Small random IOs are measured by IOPS, while large IOs are measured by bandwidth. Larger IOs reduce IOPS but increase throughput; storage selection (SSD vs. SAS) depends on the dominant IO size of the workload.

Cache Acceleration Principles

Cache accelerates write paths through write‑back buffering, write hit detection, and write merging. For reads, cache provides hit acceleration and full‑cache scenarios where no disk access is needed, delivering the maximum possible IOPS of a storage system.

Conclusion

Understanding IO aggregation, RAID penalties, business‑model IO patterns, FC bandwidth limits, read/write ratios, and cache behavior enables accurate storage performance evaluation and informed configuration decisions for diverse enterprise workloads.