Unlock Faster Linux I/O: Master Zero‑Copy Techniques and Boost Performance

Linux zero‑copy technology is a fundamental skill for cloud computing, as mastering Linux greatly aids learning in this field.

This article summarizes several common zero‑copy techniques on Linux, with reference links provided at the end.

Why Zero‑Copy Is Needed

Traditional Linux I/O (read/write) copies data between user and kernel space (copy_to_user/copy_from_user), which reduces disk I/O but incurs heavy CPU usage due to frequent user‑kernel switches; studies show that copying can consume up to 57.1% of packet processing time in the kernel stack.

What Is Zero‑Copy

Zero‑copy aims to avoid data copying to relieve CPU pressure. Linux zero‑copy methods fall into two categories: removing unnecessary copies for specific scenarios, and optimizing the overall copy process. It is more a design philosophy than a literal “zero” copy.

Zero‑Copy Methods

Original Data Copy Operations

Typical data flow when an application reads from a disk file and sends it over the network involves multiple copies: disk → kernel buffer → user buffer → socket buffer → NIC.

while((n = read(diskfd, buf, BUF_SIZE)) > 0)

    write(sockfd, buf , n);

This process includes at least four copies, two of which are DMA transfers that bypass the CPU, while the remaining two are CPU‑bound copies.

Method 1: User‑Space Direct I/O

Allows applications or libraries in user space to access hardware directly, bypassing the kernel for data transfer, which greatly improves performance.

Drawbacks:

Only suitable for applications that do not require kernel buffering, such as self‑caching applications like DBMS.

Direct disk I/O can waste resources due to CPU‑disk speed mismatch; combining with asynchronous I/O is needed.

Method 2: mmap

Using mmap instead of read eliminates one copy.

buf = mmap(diskfd, len);

write(sockfd, buf, len);

Data is DMA‑copied to a kernel buffer, then shared with the application, avoiding a user‑space copy. The subsequent write copies from kernel buffer to socket buffer, then DMA to NIC.

Drawbacks:

If another process truncates the mapped file, write may receive a SIGBUS and terminate.

Solution: use file lease locks to receive a real‑time signal before the write fails.

Method 3: sendfile

Since Linux 2.1, sendfile transfers data entirely in kernel space, avoiding user‑space copies.

#include<sys/sendfile.h>

ssize_t sendfile(int out_fd, int in_fd, off_t *offset, size_t count);

sendfile

requires the input file to be mmap‑able and the output to be a socket, eliminating user‑space involvement and providing built‑in error handling.

Drawback: only works for data that does not need user‑space processing.

Method 4: DMA‑Assisted sendfile

Standard sendfile still copies data from kernel buffer to socket buffer; DMA‑assisted sendfile copies only buffer descriptors, letting the DMA engine move data directly from kernel buffer to the NIC.

Drawbacks:

Requires hardware and driver support.

Only applicable for copying data from a file to a socket.

Method 5: splice

splice

removes the file‑type limitation of sendfile, allowing data transfer between any two file descriptors, but at least one must be a pipe.

#define _GNU_SOURCE
#include <fcntl.h>

ssize_t splice(int fd_in, loff_t *off_in, int fd_out, loff_t *off_out, size_t len, unsigned int flags);

splice

uses Linux pipe buffers and can increase throughput by 30‑70% while halving CPU usage.

Drawbacks:

Only suitable for programs that do not need user‑space processing.

At least one descriptor must be a pipe.

Method 6: Copy‑On‑Write

When multiple processes share a kernel buffer, a write triggers a copy of the page to a private address space, avoiding unnecessary copies for read‑only data.

Drawback: Requires MMU support to detect write attempts and allocate new pages.

Method 7: Buffer Sharing (Fast Buffer – fbuf)

Each process maintains a buffer pool mapped into both user and kernel address spaces, allowing direct sharing without copies.

Drawbacks:

Requires tight cooperation among applications, network software, and drivers.

API redesign is still experimental.

High‑Performance Network I/O Framework – netmap

Netmap uses shared memory to provide a high‑performance packet I/O framework without modifying the OS or needing special hardware; it maps kernel packet pools into user space via mmap, allowing user‑space programs to access rings directly.

Summary

Zero‑copy embodies an optimization mindset; techniques include direct I/O, mmap, sendfile, DMA‑assisted sendfile, splice, buffer sharing, and copy‑on‑write, each with specific use cases and limitations.

Author: 卡巴拉的树 Source: https://www.cnblogs.com/bakari/p/11275735.html