Why Kernel‑User Mode Switching Slows I/O and How DMA & Zero‑Copy Boost Performance

Storage Media Performance

The diagram shows the read/write speed of various storage media from disk to memory; the closer a medium is to the CPU, the faster its performance. Understanding these speeds highlights the benefit of zero‑copy for high‑performance I/O systems.

Kernel Mode and User Mode

Kernel mode (kernel space) can access all memory and control peripheral devices, while user mode (user space) has restricted memory access. The kernel acts as a privileged manager, and user processes run with limited rights.

How Kernel and User Modes Control Data Transfer

When process a on computer A sends a file to process b on computer B, the I/O path involves four steps: (1) a system call read copies data from disk to the kernel’s page cache (kernel mode); (2) the data is copied from the page cache to the user‑space buffer (user mode); (3) the data moves from the user buffer to the kernel’s socket buffer (kernel mode); (4) the socket buffer is transmitted over the network (user mode). This results in four context switches.

What Is DMA?

Direct Memory Access (DMA) is a hardware technique that lets I/O devices transfer data to or from memory without CPU involvement, reducing CPU load and improving throughput.

Zero‑Copy Technique

Zero‑copy combines DMA and PageCache so that data is transferred directly between disk, kernel buffers, and network interfaces without CPU‑mediated copies, cutting the number of copies and context switches roughly in half and at least doubling transfer performance.

PageCache

PageCache is the kernel’s disk cache. When a read request arrives, the kernel first checks if the data is already cached (cache hit); otherwise it reads from disk (cache miss) and stores the data in PageCache. Writes are performed to the cache and marked dirty; the kernel later flushes dirty pages to disk based on time or memory‑usage thresholds. PageCache also implements read‑ahead prefetch to reduce random‑access latency.

Advantages: faster data access, fewer disk I/O operations, higher I/O throughput. Disadvantages: consumes additional RAM, can cause swap pressure, and lacks a clean API for applications, leading to sub‑optimal cache usage for large files.

Zero‑Copy Implementation Methods

Two common approaches are:

mmap + write: mmap maps the kernel buffer into user space, eliminating one copy compared to read/write.

sendfile: a Linux system call that copies data from a file descriptor directly to a socket, removing both read and write calls and reducing context switches. With SG‑DMA support, sendfile can further avoid copying data into the socket buffer.

Large File Transfer

For large files, asynchronous I/O combined with direct I/O is preferred over zero‑copy, as it avoids the overhead of PageCache and minimizes blocking.

Typical tuning parameters (viewable via sysctl -a | grep dirty) include vm.dirty_background_ratio, vm.dirty_ratio, vm.dirty_expire_centisecs, vm.dirty_writeback_centisecs, and vm.swappiness. Adjust these based on CPU cores, memory size, disk type, and network bandwidth to optimize I/O performance.