How Zero‑Copy, DMA, and RDMA Supercharge Data Transfer in Linux

Part 1 Zero‑Copy Technology

Traditional data transfer moves data through multiple buffers: disk → kernel buffer → user buffer → socket buffer → NIC, causing several CPU‑intensive copies and context switches. Each copy consumes CPU cycles and reduces throughput, especially under high concurrency.

1.1 Traditional Data Copy Problems

Switch from user mode to kernel mode (read system call).

Disk data is DMA‑copied into the kernel buffer.

Kernel buffer is copied to the user buffer.

Switch back to user mode after read returns.

Switch to kernel mode again for write.

User buffer is copied to the socket buffer.

Kernel DMA‑copies socket buffer to the NIC.

Switch back to user mode after write returns.

Four mode switches and four copies (two DMA, two CPU) dramatically increase latency and CPU load.

1.2 Definition and Core Idea of Zero‑Copy

Zero‑copy does not eliminate all copies; it reduces redundant copies between user and kernel space and minimizes CPU involvement. Its goals are to lower copy count, reduce context switches, and avoid unnecessary memory duplication.

Minimize data copies (ideally 0 CPU copies).

Reduce system‑call count, cutting user↔kernel switches.

Prevent duplicate storage in user space.

Example: the Linux sendfile system call transfers data directly from the kernel cache to the socket buffer, bypassing user space.

Part 2 DMA – The Hardware Foundation of Zero‑Copy

2.1 What Is RDMA?

Remote Direct Memory Access (RDMA) allows a computer to read/write memory on a remote machine without CPU involvement, dramatically lowering latency for high‑performance computing, data‑center, and cloud workloads.

2.2 How DMA Works

DMA lets hardware devices move data between memory and peripherals without CPU intervention.

Initialization: CPU programs the DMA controller with source, destination, and length.

Request: Device signals the DMA controller.

Bus arbitration: DMA gains control of the system bus.

Transfer: DMA moves data directly between device and memory.

Completion: DMA raises an interrupt; CPU resumes other work.

2.3 Role of DMA in Zero‑Copy

When receiving network packets, the NIC can DMA data straight into the kernel buffer, avoiding a copy to user space. When reading from disk, the disk controller DMA‑writes data into the kernel PageCache, and the CPU only initiates the transfer.

Part 3 PageCache – The Software Accelerator

3.1 PageCache Mechanism

PageCache stores disk pages (typically 4 KB) in RAM. On a read, the kernel first checks the cache; a hit returns data directly from memory, avoiding disk I/O. A miss triggers a DMA‑driven read into the cache, after which the data is served.

3.2 Interaction Between PageCache and Zero‑Copy

Zero‑copy system calls such as sendfile first ensure the file is in PageCache, then copy from PageCache to the socket buffer, eliminating an extra user‑space copy.

Part 4 Zero‑Copy Implementation Techniques

4.1 mmap + write

Memory‑map a file with mmap, then write the mapped region directly to a socket. The data resides in kernel PageCache, so only one copy (PageCache → socket) occurs.

Call mmap to map the file.

Read data into PageCache.

Call write to move data from PageCache to the socket.

DMA transfers the socket buffer to the NIC.

4.2 sendfile

sendfile

transfers data from a file descriptor to a socket descriptor entirely within the kernel. Early Linux kernels used simple copy; later kernels employ scatter‑gather DMA for true zero‑copy.

4.3 splice and tee

splice

moves data between two file descriptors via an internal pipe without user‑space copies. tee duplicates pipe data to multiple destinations, useful for logging or broadcasting.

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <fcntl.h>
#include <assert.h>
int main(int argc, char **argv) {
    if (argc <= 2) {
        printf("usage: %s ip port
", argv[0]);
        return 1;
    }
    const char *ip = argv[1];
    int port = atoi(argv[2]);
    int sock = socket(AF_INET, SOCK_STREAM, 0);
    assert(sock >= 0);
    int reuse = 1;
    setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &reuse, sizeof(reuse));
    struct sockaddr_in address;
    bzero(&address, sizeof(address));
    address.sin_family = AF_INET;
    address.sin_port = htons(port);
    inet_pton(AF_INET, ip, &address.sin_addr);
    assert(bind(sock, (struct sockaddr*)&address, sizeof(address)) != -1);
    assert(listen(sock, 5) != -1);
    struct sockaddr_in client;
    socklen_t client_len = sizeof(client);
    int connfd = accept(sock, (struct sockaddr*)&client, &client_len);
    if (connfd < 0) {
        printf("errno: %s
", strerror(errno));
    } else {
        int pipefd[2];
        assert(pipe(pipefd) != -1);
        assert(splice(connfd, NULL, pipefd[1], NULL, 32768, SPLICE_F_MORE | SPLICE_F_MOVE) != -1);
        assert(splice(pipefd[0], NULL, connfd, NULL, 32768, SPLICE_F_MORE | SPLICE_F_MOVE) != -1);
        close(connfd);
    }
    close(sock);
    return 0;
}

Part 5 Application Scenarios

5.1 Network Transfer

High‑performance web servers (e.g., Nginx) use sendfile to serve static files directly from PageCache to the network, reducing CPU load and increasing throughput. CDNs also rely on zero‑copy to deliver large media files efficiently.

5.2 File Processing

Large‑file read/write can be accelerated with mmap or sendfile. Database backups benefit from zero‑copy by streaming data from disk to backup storage without extra copies.

Part 6 Zero‑Copy in Java

6.1 Java NIO mmap Support

public class MmapTest {
    public static void main(String[] args) {
        try {
            FileChannel readChannel = FileChannel.open(Paths.get("./jay.txt"), StandardOpenOption.READ);
            MappedByteBuffer data = readChannel.map(FileChannel.MapMode.READ_ONLY, 0, 1024 * 1024 * 40);
            FileChannel writeChannel = FileChannel.open(Paths.get("./siting.txt"), StandardOpenOption.WRITE, StandardOpenOption.CREATE);
            writeChannel.write(data);
            readChannel.close();
            writeChannel.close();
        } catch (Exception e) {
            System.out.println(e.getMessage());
        }
    }
}

6.2 Java NIO sendfile Support

public class SendFileTest {
    public static void main(String[] args) {
        try {
            FileChannel readChannel = FileChannel.open(Paths.get("./jay.txt"), StandardOpenOption.READ);
            long len = readChannel.size();
            long pos = readChannel.position();
            FileChannel writeChannel = FileChannel.open(Paths.get("./siting.txt"), StandardOpenOption.WRITE, StandardOpenOption.CREATE);
            readChannel.transferTo(pos, len, writeChannel);
            readChannel.close();
            writeChannel.close();
        } catch (Exception e) {
            System.out.println(e.getMessage());
        }
    }
}

6.3 Example: Zero‑Copy File Server

import java.io.FileInputStream;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.channels.FileChannel;
import java.nio.channels.ServerSocketChannel;
import java.nio.channels.SocketChannel;

public class ZeroCopyFileServer {
    public static void main(String[] args) throws IOException {
        int port = 8888;
        ServerSocketChannel server = ServerSocketChannel.open();
        server.bind(new InetSocketAddress(port));
        while (true) {
            SocketChannel client = server.accept();
            String filePath = "your_file_path_here"; // replace with actual path
            try (FileInputStream fis = new FileInputStream(filePath);
                 FileChannel fc = fis.getChannel()) {
                long transferred = fc.transferTo(0, fc.size(), client);
                System.out.println("Transferred " + transferred + " bytes.");
            }
            client.close();
        }
    }
}

This server uses FileChannel.transferTo (internally the sendfile syscall) to stream a file to a client without copying data into user space.