Understanding Zero-Copy in Java: Concepts, Implementations, and Netty

Preface

Zero‑copy means data does not need to be copied back and forth between user space and kernel space, greatly improving system performance. The term appears frequently in Java NIO, Netty, Kafka, RocketMQ and other frameworks as a key performance feature. The article starts with basic I/O concepts and then analyses zero‑copy.

I/O Concepts

1. Buffer

A buffer is the foundation of all I/O. Performing I/O means moving data into or out of a buffer. When a process issues a read request, the kernel first checks if the required data already resides in kernel space; if so, it copies the data to the process buffer. Otherwise the kernel commands the disk controller to read the data directly into the kernel read buffer via DMA, after which the kernel copies the data to the process buffer. Write operations copy data from the user buffer to the kernel socket buffer, then DMA transfers it to the NIC. Zero‑copy aims to eliminate these copies, using either mmap+write or sendfile approaches.

2. Virtual Memory

Modern operating systems use virtual memory, mapping virtual addresses to physical memory. This allows multiple virtual addresses to refer to the same physical page and enables a virtual address space larger than physical RAM. By mapping kernel and user virtual addresses to the same physical page, DMA can fill a buffer visible to both kernel and user space, reducing copies.

3. mmap+write Method

Using mmap+write replaces the traditional read‑then‑write flow. mmap maps a file into the process address space, establishing a one‑to‑one relationship between file offsets and virtual addresses, thus eliminating the kernel read buffer copy to user space. However, a copy from the kernel read buffer to the kernel socket buffer may still occur.

4. sendfile Method

The sendfile system call, introduced in kernel 2.1, simplifies data transfer between two channels by performing the copy entirely in kernel space, reducing both data copies and context switches. Later kernel improvements recorded the data descriptor in the socket buffer, eliminating even the remaining kernel‑space copy.

Java Zero‑Copy

1. MappedByteBuffer

Java NIO's FileChannel.map() creates a virtual memory mapping between a file and a MappedByteBuffer, which extends ByteBuffer. The buffer's data resides on disk but can be accessed as if it were in memory. Example code reads a file via a mapped buffer and processes its content.

public class MappedByteBufferTest {
    public static void main(String[] args) throws Exception {
        File file = new File("D://db.txt");
        long len = file.length();
        byte[] ds = new byte[(int) len];
        MappedByteBuffer mappedByteBuffer = new FileInputStream(file).getChannel()
            .map(FileChannel.MapMode.READ_ONLY, 0, len);
        for (int offset = 0; offset < len; offset++) {
            byte b = mappedByteBuffer.get();
            ds[offset] = b;
        }
        Scanner scan = new Scanner(new ByteArrayInputStream(ds)).useDelimiter(" ");
        while (scan.hasNext()) {
            System.out.print(scan.next() + " ");
        }
    }
}

The map() method signature:

public abstract MappedByteBuffer map(FileChannel.MapMode mode, long position, long size) throws IOException;

Parameters: MapMode (READ_ONLY, READ_WRITE, PRIVATE), position (start offset), and size (length). The source code shows native address acquisition, garbage‑collection handling, and creation of a DirectByteBuffer instance.

2. DirectByteBuffer

DirectByteBuffer

extends MappedByteBuffer and allocates memory outside the JVM heap, avoiding GC overhead. It can also be created manually:

ByteBuffer directByteBuffer = ByteBuffer.allocateDirect(100);

3. Channel‑to‑Channel Transfer

FileChannel.transferTo()

moves data directly between channels without an intermediate user‑space buffer. Example:

public class ChannelTransfer {
    public static void main(String[] args) throws Exception {
        String[] files = new String[1];
        files[0] = "D://db.txt";
        catFiles(Channels.newChannel(System.out), files);
    }
    private static void catFiles(WritableByteChannel target, String[] files) throws Exception {
        for (int i = 0; i < files.length; i++) {
            FileInputStream fis = new FileInputStream(files[i]);
            FileChannel channel = fis.getChannel();
            channel.transferTo(0, channel.size(), target);
            channel.close();
            fis.close();
        }
    }
}

Method signature:

public abstract long transferTo(long position, long count, WritableByteChannel target) throws IOException;

This avoids copying data into a user buffer; both source and destination have kernel buffers, and the kernel moves data directly between them.

Netty Zero‑Copy

Netty provides zero‑copy buffers using CompositeChannelBuffer and Slice. A composite buffer aggregates multiple ChannelBuffer instances without copying their contents, enabling efficient assembly of HTTP messages from TCP fragments.

public class CompositeChannelBuffer extends AbstractChannelBuffer {
    private final ByteOrder order;
    private ChannelBuffer[] components;
    private int[] indices;
    private int lastAccessedComponentId;
    private final boolean gathering;
    public byte getByte(int index) {
        int componentId = componentId(index);
        return components[componentId].getByte(index - indices[componentId]);
    }
    ...
}

Other Zero‑Copy Uses

RocketMQ writes messages sequentially to a commit log and uses mmap+write for consumer reads. Kafka also employs sendfile for transferring persisted data to the network.

Summary

Zero‑copy in Java essentially means operating on object references rather than duplicating data; each reference points to a single underlying object, so modifications are reflected everywhere without creating multiple copies.