Why Disk Space Stays Full After Deleting Files? Inside Linux VFS and Inodes

Background

Sometimes the disk appears full even though there is still free space when checking individual files.

Investigation Steps

1. Run df to view disk usage; it shows the filesystem is 100% used.

Filesystem     Type      Size  Used Avail Use% Mounted on
/dev/vda1      ext4       30G    30G 0      100% /
devtmpfs       devtmpfs  489M     0  489M   0% /dev
tmpfs          tmpfs     497M     0  497M   0% /dev/shm
tmpfs          tmpfs     497M   50M  447M  11% /run
tmpfs          tmpfs     497M     0  497M   0% /sys/fs/cgroup

2. Run du -h --max-depth=1 /home to sum directory sizes; it shows about 10 GB missing.

-bash-4.2$ du -h --max-depth=1 /home
/home 16M
/home/logs 11G
/home/serverdog 11G
/home

3. The missing space is caused by deleted files that are still held open by processes. Using lsof reveals such files, and restarting the offending processes frees the space.

-bash-4.2# lsof | grep delete
mysqld 2470 mysql 4u REG 253,1 0 523577 /var/tmp/ibfTeQFn (deleted)
mysqld 2470 mysql 5u REG 253,1 0 523579 /var/tmp/ibaHcIdW (deleted)
mysqld 2470 mysql 6u REG 253,1 0 523581 /var/tmp/ibLjiALu (deleted)
mysqld 2470 mysql 7u REG 253,1 0 523585 /var/tmp/ibCFnzTB (deleted)
mysqld 2470 mysql 11u REG 253,1 0 523587 /var/tmp/ibCjuqva (deleted)

Virtual File System (VFS)

The VFS is the entry point for all file operations, providing an abstraction layer between user‑space libraries and specific file‑system implementations.

Common File Model

The VFS defines a common file model consisting of several objects:

Superblock object – holds filesystem metadata in memory and on disk.

Inode object – represents a file’s metadata; each inode has a unique number.

File object – created when a file is opened, linking a process to an inode.

Dentry object – represents a directory entry linking a name to an inode.

Directory Tree Construction

The root filesystem is mounted first; other filesystems are mounted as sub‑directories, forming a unified tree.

start_kernel
  vfs_caches_init
    mnt_init
      init_rootfs   // register rootfs
      init_mount_tree // mount rootfs
  …
  rest_init
  kernel_thread(kernel_init, NULL, CLONE_FS);

Soft Link vs Hard Link

A soft link is a regular file containing the path of another file; a hard link points directly to the same inode. When the inode’s link count (i_nlink) reaches zero, the file is removed.

File & Process Management

Each process that opens a file gets its own file object, while hard links share the same inode.

Inodes have both in‑memory and on‑disk representations. Linux uses a write‑back strategy: an inode is loaded into memory when a file is opened and written back to disk when it is no longer in use.

* "in_use" - valid inode, i_count > 0, i_nlink > 0
* "dirty"  - as "in_use" but also dirty
* "unused" - valid inode, i_count = 0

Open and close operations adjust the inode’s i_count via iget and iput. When i_count drops to zero and i_nlink is zero, the inode and its disk blocks are freed.

File & Disk Management

The rm command ultimately calls unlink, which removes a directory entry and decrements the inode’s link count. The kernel updates the parent directory’s inode and the target file’s inode accordingly.

# dtruss rm tmp
...
geteuid(0x0, 0x0, 0x0) = 0 0
ioctl(0x0, 0x4004667A, 0x7FFEE06F09C4) = 0 0
lstat64("tmp\0", 0x7FFEE06F0968, 0x0) = 0 0
access("tmp\0", 0x2, 0x0) = 0 0
unlink("tmp\0", 0x0, 0x0) = 0 0

Summary

The core of Linux file management is the inode index, not the file data itself. Separating index from data is key to understanding the filesystem.

Because Linux uses a write‑back cache, memory must be released before disk space can be reclaimed.

The lsof command lists open files, allowing you to locate deleted files that are still consuming space.