Understanding Linux I/O Flush: fsync, fdatasync, sync, O_DIRECT, O_SYNC, REQ_PREFLUSH and REQ_FUA

The guide introduces Linux I/O flush concepts, starting with the system calls fsync() , fdatasync() and sync() . fsync(int fd) forces all buffered data and metadata of a file descriptor to be written to non‑volatile storage, while fdatasync(int fd) flushes only the data and essential metadata. sync() flushes all kernel buffers, including data blocks, inode tables and the superblock.

It then describes the open‑file flags O_DIRECT and O_SYNC . O_DIRECT bypasses the page cache, causing I/O to go directly between user buffers and the device, which may reduce performance but avoids double caching. O_SYNC ensures that each write is completed on the device before the call returns, guaranteeing data safety.

The article also covers the block‑layer request flags REQ_PREFLUSH and REQ_FUA . REQ_PREFLUSH forces a cache flush before the I/O starts, while REQ_FUA (Forced Unit Access) guarantees that the I/O completes only after the data has been committed to non‑volatile storage.

Experimental verification is performed with dd on a /dev/bcache0 device. Various flag combinations are tested, and the kernel logs show the resulting bio->bi_opf values. Example commands:

# dd if=/dev/zero of=/dev/bcache0 oflag=direct bs=8k count=1

# dd if=/dev/zero of=/dev/bcache0 oflag=direct,sync bs=8k count=1

# dd if=/dev/zero of=/dev/bcache0 oflag=direct,sync conv=fdatasync bs=8k count=1

Kernel log excerpts illustrate how different flag sets map to REQ_OP_WRITE , REQ_SYNC , REQ_IDLE , REQ_FUA and REQ_PREFLUSH bits, e.g., bi_opf 165889 = REQ_OP_WRITE | REQ_SYNC | REQ_IDLE | REQ_FUA .

A table of decimal, binary and symbolic flag values is provided, showing how combinations such as 2409, 4096, 34817, 399361, etc., correspond to specific REQ_OP_* and REQ_* flags.

The source also includes the kernel enum definitions for request operations and flag bits:

enum req_opf {
    REQ_OP_READ = 0,
    REQ_OP_WRITE = 1,
    REQ_OP_FLUSH = 2,
    REQ_OP_DISCARD = 3,
    REQ_OP_SECURE_ERASE = 5,
    REQ_OP_ZONE_RESET = 6,
    REQ_OP_WRITE_SAME = 7,
    REQ_OP_ZONE_RESET_ALL = 8,
    REQ_OP_WRITE_ZEROES = 9,
    REQ_OP_SCSI_IN = 32,
    REQ_OP_SCSI_OUT = 33,
    REQ_OP_DRV_IN = 34,
    REQ_OP_DRV_OUT = 35,
    REQ_OP_LAST,
};

enum req_flag_bits {
    __REQ_FAILFAST_DEV = REQ_OP_BITS,
    __REQ_FAILFAST_TRANSPORT,
    __REQ_FAILFAST_DRIVER,
    __REQ_SYNC,
    __REQ_META,
    __REQ_PRIO,
    __REQ_NOMERGE,
    __REQ_IDLE,
    __REQ_INTEGRITY,
    __REQ_FUA,
    __REQ_PREFLUSH,
    __REQ_RAHEAD,
    __REQ_BACKGROUND,
    __REQ_NOWAIT,
    __REQ_NOWAIT_INLINE,
    ...
};

Finally, practical commands for checking and toggling the hard‑disk write cache are shown:

# hdparm -W /dev/sda

# hdparm -W 0 /dev/sda   # disable write cache

# hdparm -W 1 /dev/sda   # enable write cache

These examples help readers understand how different flush mechanisms affect data durability and performance on Linux systems.