Unlocking Linux File System Secrets: Inodes, VFS, and I/O Performance

1 前言

Like CPU and memory, disk and file system management are core OS functions. Disks provide persistent storage, and file systems organize files in a tree structure. This section introduces how disks and file systems work and the metrics to evaluate their performance.

2 索引节点和目录项

File systems organize files on storage devices; different organization methods create different file systems. In Linux, everything is a file, including block devices, sockets, and pipes, which are managed through a unified file system.

Linux assigns two data structures to each file: 索引节点 (index node) and 目录项 (directory entry). They record metadata and directory structure.

索引节点 (inode) stores metadata such as inode number, size, permissions, timestamps, and data locations. Each inode corresponds to a file and occupies disk space.

目录项 (dentry) stores the file name, inode pointer, and relationships to other entries. Dentries are cached in memory by the kernel.

Inodes uniquely identify files, while dentries maintain the hierarchical structure. Multiple dentries can point to the same inode, enabling hard links.

Disk I/O operates on sectors (typically 512 B), but to improve efficiency, sectors are grouped into logical blocks (commonly 4 KB). This logical block is the minimal unit for file system data management.

Below is a diagram illustrating the relationship among dentries, inodes, logical blocks, and the superblock.

Two important points:

Directory entries are memory caches, while inodes reside on disk but are also cached in memory to accelerate access.

During file system formatting, the disk is divided into three regions: superblock, inode area, and data block area.

The superblock stores the file system state, the inode area stores inodes, and the data block area stores file data.

3 虚拟文件系统

The four basic elements—directory entry, inode, logical block, and superblock—form the core of Linux file systems. To support various file systems, Linux introduces the Virtual File System (VFS) layer as an abstraction.

VFS defines a set of data structures and standard interfaces that all file systems implement, allowing user processes and kernel subsystems to interact through a unified API.

The following diagram shows the relationship among system calls, VFS, cache, file systems, and block storage.

Linux supports many file systems (e.g., Ext4, XFS, NFS). Based on storage location, they fall into three categories:

Disk‑based file systems store data on local disks (e.g., Ext4, XFS, OverlayFS).

Memory‑based file systems (virtual file systems) reside entirely in RAM, such as /proc and /sys.

Network file systems access remote storage (e.g., NFS, SMB, iSCSI).

These file systems must be mounted at a VFS mount point before their files become accessible.

4 文件系统 I/O

After mounting, VFS provides standard file access interfaces via system calls. For example, the cat command uses open(), read(), and write():

int open(const char *pathname, int flags, mode_t mode);
ssize_t read(int fd, void *buf, size_t count);
ssize_t write(int fd, const void *buf, size_t count);

File I/O can be classified in four ways:

1. Buffered vs. Unbuffered I/O

Buffered I/O uses the standard library cache before invoking system calls.

Unbuffered I/O calls the kernel directly, bypassing the library cache.

Both ultimately rely on system calls and the page cache.

2. Direct vs. Indirect I/O

Direct I/O skips the page cache (use O_DIRECT flag).

Indirect I/O goes through the page cache before reaching the disk.

Direct I/O is common in database workloads; skipping the file system entirely is called raw I/O.

3. Blocking vs. Non‑blocking I/O

Blocking I/O suspends the thread until the operation completes.

Non‑blocking I/O returns immediately; the application polls or receives events.

Setting O_NONBLOCK on pipes or sockets enables non‑blocking mode.

4. Synchronous vs. Asynchronous I/O

Synchronous I/O waits for the operation to finish before returning.

Asynchronous I/O returns immediately; completion is notified via signals such as SIGIO or SIGPOLL.

Flags like O_SYNC, O_DSYNC, and O_ASYNC control these behaviors.

5 性能观测

To monitor file system performance, start a terminal on the server.

容量

Use df to check disk usage:

df /dev/vda1
文件系统   1K-块   已用   可用   已用%   挂载点
/dev/vda1 104846316 28228044 76618272 27% /

Adding -h makes the output human‑readable:

df -h /dev/vda1
文件系统   容量 已用 可用 已用% 挂载点
/dev/vda1 100G 27G 74G 27% /

Inode usage can be inspected with -i:

df -h -i /dev/vda1
文件系统   Inode 已用(I) 可用(I) 已用(I)% 挂载点
/dev/vda1 50M 162K 50M 1% /

Running out of inodes usually means many small files.

缓存

Use free or vmstat to view page cache size. The /proc/meminfo file shows cached memory:

cat /proc/meminfo | grep -E "SReclaimable|Cached"
Cached:          2014100 kB
SwapCached:        5316 kB
SReclaimable:    216128 kB

Directory entry and inode caches are managed by the slab allocator. Detailed slab statistics are in /proc/slabinfo:

cat /proc/slabinfo | grep -E '^#|dentry|inode'
# name <active_objs> <num_objs> <objsize> <objperslab> <pagesperslab> : tunables ...
ovl_inode 66 66 736 22 4 : tunables ...
...

The dentry line shows the dentry cache, and inode_cache shows the VFS inode cache.

For a concise view, use slabtop:

# Press c to sort by cache size, a to sort by active objects
slabtop
Active / Total Objects (% used)    : 991123 / 1087653 (91.1%)
...

In the example, dentry and inode_cache dominate slab usage, but together they consume only about 60 MB.

6 总结

The file system organizes and manages files on storage devices. Linux abstracts various file system implementations through the Virtual File System (VFS) layer, providing a uniform interface for user processes and kernel subsystems.

VFS defines common data structures and interfaces, while page cache, dentry cache, and inode cache mitigate disk latency for applications.

Understanding these components and monitoring tools helps diagnose and optimize file system performance.