Why Traditional System Call I/O Slows Down Linux Applications—and How to Optimize It

Traditional System Call I/O

In Linux, the traditional way to access files and network sockets is through the write() and read() system calls. read() copies data from a file into a kernel buffer, and write() copies data from a kernel buffer to a network port.

The diagram shows that traditional I/O requires two CPU copies, two DMA copies, and four context switches.

CPU copy : Data transfer handled directly by the CPU, occupying CPU resources.

DMA copy : The CPU instructs the DMA controller to move data, freeing the CPU.

Context switch : Switching between user space and kernel space on entry and exit of the system call.

Read Operation

If the requested data is already in the process's page memory, read() returns it directly; otherwise the data is loaded from disk into the kernel read buffer and then copied to user space.

read(file_fd, tmp_buf, len);

Traditional read() triggers two context switches, one DMA copy and one CPU copy.

User process calls read() – context switches to kernel space.

CPU uses DMA to copy data from memory or disk to the kernel read buffer.

CPU copies data from the read buffer to the user buffer.

Context switches back to user space and returns.

Write Operation

When an application calls write(), data is first copied from the user page cache to the kernel socket buffer, then from the socket buffer to the NIC for transmission.

Traditional write() also causes two context switches, one CPU copy and one DMA copy.

User process calls write() – context switches to kernel space.

CPU copies data from the user buffer to the kernel socket buffer.

CPU uses DMA to move data from the socket buffer to the NIC.

Context switches back to user space and returns.

Network I/O

Disk I/O

High‑Performance I/O Optimizations

Zero‑copy techniques.

Multiplexing.

PageCache usage.

PageCache is the OS file cache that reduces disk I/O by storing data in memory pages, making sequential reads and writes almost as fast as memory access.

PageCache Read Strategy

When a process issues read(), the kernel first checks if the data is in PageCache; if not, it reads the required blocks from disk into PageCache (typically a few pages) and then serves the request.

If present, the data is read directly from PageCache.

If absent, the kernel schedules an I/O operation to fetch the data from disk into PageCache.

PageCache Write Strategy

When write() is called, data is first written to PageCache and marked as “dirty”. A flusher thread later writes dirty pages back to disk, triggered by low free memory, dirty‑page age, or explicit sync()/fsync() calls.

Free memory falls below a threshold.

Dirty pages reside longer than a threshold.

Application invokes sync() or fsync().

I/O Buffering

The diagram shows the flow from user data to disk, illustrating the layers of buffers in Linux.

StdIO Buffering

The C stdio library provides its own user‑space buffers to reduce the number of system calls. Functions like fflush() and setbuf() control this buffering.

Kernel Buffer Cache

Between system calls and the actual disk lies the kernel buffer cache (PageCache for file contents and BufferCache for raw device blocks).

Direct I/O and mmap

Direct I/O bypasses PageCache, allowing data to be transferred directly between user space and the block device via DMA, which reduces copies but requires page‑aligned buffers and I/O sizes that are multiples of the device’s block size.

mmap maps PageCache directly into the process’s address space, eliminating the second copy from PageCache to user space.

Using these advanced mechanisms demands careful application‑level design to handle alignment constraints and to maintain data consistency.