Why Does Disk Space Vanish? Uncovering Linux’s VFS and File Deletion Mysteries

Background

Sometimes disk space appears full with df, yet du shows plenty of free space.

-bash-4.2$ df -Th
Filesystem     Type      Size  Used Avail Use% Mounted on
/dev/vda1      ext4       30G    30G 0      100% /
devtmpfs       devtmpfs  489M     0  489M   0% /dev
tmpfs          tmpfs     497M     0  497M   0% /dev/shm
tmpfs          tmpfs     497M   50M  447M  11% /run
tmpfs          tmpfs     497M     0  497M   0% /sys/fs/cgroup

Running du -h --max-depth=1 /home reveals only about 22 GB used, leaving ~10 GB mysteriously missing.

-bash-4.2$ du -h --max-depth=1 /home
16M /home
11G /home/logs
11G /home/serverdog

The cause is that deleted files are still held open by processes, so their space is not released. The lsof command shows such open‑deleted files; restarting the offending processes frees the space.

-bash-4.2# lsof | grep delete
2470  mysql  4u  REG  253,1 0 523577 /var/tmp/ibfTeQFn (deleted)
2470  mysql  5u  REG  253,1 0 523579 /var/tmp/ibaHcIdW (deleted)
2470  mysql  6u  REG  253,1 0 523581 /var/tmp/ibLjiALu (deleted)
2470  mysql  7u  REG  253,1 0 523585 /var/tmp/ibCFnzTB (deleted)
2470  mysql 11u  REG  253,1 0 523587 /var/tmp/ibCjuqva (deleted)

Virtual File System (VFS)

The VFS is the entry point for all file operations, providing an abstraction layer between user‑space libraries and specific file‑system implementations.

VFS defines a common file model consisting of:

Superblock object

Inode object

File object

Dentry object

File concept

Common File Model Details

Superblock object

Memory: created at mount time, stores file‑system metadata. Disk: corresponds to the filesystem control block on disk.

Inode object

Memory: created on access, holds general file information ( inode structure). Disk: stores the file control block. Each inode has a unique number identifying the file.

File object

Memory: created when a file is opened, contains information linking the open file to a process ( file structure). It exists only while the process accesses the file.

Dentry object

Memory: when a directory entry is read, VFS converts it to a dentry structure. Disk: stores directory entry information linking names to inodes.

Directory Tree Construction

The root filesystem is mounted first; subsequent file‑systems are mounted as sub‑trees, forming the complete directory hierarchy.

start_kernel
  vfs_caches_init
    mnt_init
      init_rootfs   // register rootfs
      init_mount_tree // mount rootfs
  …
  rest_init
  kernel_thread(kernel_init, NULL, CLONE_FS);

Soft Link vs Hard Link

A soft link is a regular file containing the pathname of another file; a hard link points directly to the same inode, with the link count stored in the inode’s i_nlink field. When i_nlink reaches zero, no hard links reference the file.

File & Process Management

When multiple processes open the same file, each gets its own file object, while hard links may share the same inode.

Inodes have both memory and disk representations; Linux uses a write‑back cache strategy. An inode is loaded into memory on open, and written back to disk when no longer in use.

* "in_use" - valid inode, i_count > 0, i_nlink > 0
* "dirty"  - as "in_use" but also dirty
* "unused" - valid inode, i_count = 0

The open() and close() system calls create and destroy file objects, updating the inode’s i_count via iget and iput. When i_count drops to zero, the inode is released from memory; if i_nlink is also zero, the disk space is freed.

File & Disk Management

Commands like touch and rm manipulate files; rm ultimately performs an unlink, removing the directory entry and decrementing the inode’s link count.

# dtruss rm tmp
...
unlink("tmp", 0, 0) = 0

Deletion steps:

Process uses open to obtain the parent directory’s file object. iget increments the directory inode’s count.

Read directory data and convert entries to dentry objects.

Increase the target file’s inode count via iget.

Remove the directory entry.

Decrement the file inode’s i_nlink. iput decrements the file inode’s i_count; if zero, memory is released, and if i_nlink is zero, disk space is reclaimed.

Finally, iput on the directory inode completes the operation.

Summary

Index vs Data

The core of file‑system, file‑disk, and process management is the file’s index (inode), not the file data itself; separating data from its index is key to understanding the system.

Cache Strategy

Linux’s write‑back policy means that memory must be released before the corresponding disk space can be reclaimed.

Why lsof ?

lsof

lists open files, allowing you to locate deleted files that are still held open and therefore not yet freed.