How Traditional Linux System Calls Affect I/O Performance and What Modern Techniques Can Do

Traditional System Call I/O

In Linux, the classic way to access data is through the write() and read() system calls. read() copies data from a file into a buffer, and write() sends the buffer to a network socket.

read(file_fd, tmp_buf, len);
write(socket_fd, tmp_buf, len);

The process involves two CPU copies, two DMA copies, and four context switches.

CPU copy: Data is moved directly by the CPU, occupying CPU resources.

DMA copy: The CPU instructs the DMA controller to transfer data, freeing the CPU.

Context switch: Transition between user mode and kernel mode for each system call.

Read Operation

If the requested data is already in the process's page cache, it is read directly from memory. Otherwise the kernel loads the data into a read buffer, then copies it to the user page cache.

Traditional read() triggers two context switches, one DMA copy and one CPU copy.

User process calls read() → kernel space.

CPU uses DMA to move data to kernel read buffer.

CPU copies data from read buffer to user buffer.

Context switches back to user space and returns.

Write Operation

When write() is called, data is first copied from the user buffer to the kernel socket buffer, then the kernel copies it to the NIC for transmission.

write(socket_fd, tmp_buf, len);

Traditional write() also causes two context switches, one CPU copy and one DMA copy.

User process calls write() → kernel space.

CPU copies data to socket buffer.

CPU uses DMA to transfer data from socket buffer to NIC.

Context switches back to user space and returns.

High‑Performance I/O Optimizations

Zero‑copy techniques.

Multiplexing (I/O multiplexing).

PageCache (page cache) mechanism.

PageCache caches file data in memory, reducing disk I/O. When a read request arrives, the kernel first checks the page cache; if present, it serves data from memory, otherwise it reads the required blocks from disk into the cache.

Write operations initially write to the page cache, marking pages as “dirty”. A background flusher periodically writes dirty pages back to disk, or they are flushed when memory is low, when they stay dirty too long, or when the application calls sync() / fsync().

Linux I/O Stack

Filesystem layer: write() copies user data to the filesystem cache.

Block layer: manages block device queues, merges and schedules I/O requests.

Device layer: interacts with the hardware via DMA.

These layers relate to Buffered I/O, mmap, and Direct I/O.

Buffered I/O vs. mmap vs. Direct I/O

Buffered I/O reads a “cold” file by loading it into the page cache, then copying to user space (two copies). mmap maps the page cache directly into user address space, eliminating the second copy. Direct I/O bypasses the page cache, transferring data directly between user buffers and the block device, reducing copies and improving throughput.

I/O Buffering

Above the kernel buffer cache there is a user‑space stdio buffer. The stdio library aggregates small reads/writes to reduce system‑call overhead and provides fflush() and setbuf() to control buffering.

Kernel buffer cache consists of PageCache (file content) and BufferCache (raw block device data).