Understanding Zero‑Copy I/O: DMA, Sendfile, mmap, and Direct I/O

When a client request triggers a file read and a socket send, the operating system normally performs four data copies and four context switches: CPU moves data from disk to page cache, from page cache to user buffer, from user buffer to socket buffer, and from socket buffer to the NIC; each read/write system call also switches between user and kernel mode.

Direct I/O, DMA, and zero‑copy techniques aim to eliminate or reduce these copies and switches.

DMA‑Assisted Data Transfer

DMA (Direct Memory Access) uses a dedicated controller to move data between memory and I/O devices without CPU involvement, allowing the CPU to issue only control signals.

After DMA is introduced, the CPU no longer copies data between page cache and socket buffers; the DMA controller handles disk‑to‑page‑cache and page‑cache‑to‑NIC transfers.

Zero‑Copy Technology

Zero‑copy means the CPU does not copy data between memory regions; instead, data is transferred directly between kernel buffers or between kernel and hardware.

sendfile

mmap

splice

Direct I/O

These methods reduce the number of copies and system calls.

sendfile

sendfile is used when data read from disk is sent over the network without modification (e.g., message queues like Kafka). It combines two system calls (read + write) into one and leverages DMA and file‑descriptor passing.

Uses DMA for disk‑to‑page‑cache and socket‑buffer‑to‑NIC transfers.

Passes the file descriptor instead of copying data.

Only one system call is needed, reducing context switches from four to two.

Note: sendfile cannot be used if the application needs to process the data (e.g., encryption) because the data never reaches user space.

mmap

mmap maps a file directly into the process’s address space, allowing the application to access the file without an explicit read system call. The mapping resides in kernel page cache, and DMA moves data between disk and the cache.

Direct I/O

Direct I/O bypasses the page cache entirely; data is transferred directly between user buffers and the storage device using DMA. Writes are persisted immediately, and reads always hit the disk.

Advantages:

Eliminates kernel‑space buffers, reducing CPU‑copy overhead.

Improves performance for large data transfers.

Disadvantages:

Requires page‑pinning, which adds overhead.

May be slower for small or random reads because each access hits the disk.

Application must manage its own buffering, increasing complexity.

Typical Use Cases

Kafka : Uses mmap for persisting logs (sequential disk I/O) and sendfile for delivering messages to consumers, allowing multiple consumers to share the same page cache without extra disk reads.

MySQL : Employs zero‑copy techniques (details in a separate article) to minimize CPU involvement during data transfer.

Summary

DMA offloads data movement between memory and I/O devices, allowing the CPU to focus on control. Linux provides several zero‑copy strategies—sendfile, mmap, splice, and Direct I/O—each reducing or eliminating copies between user and kernel space and improving I/O performance.