Why Does du Show 9 GB While ls Shows 100 GB? Understanding Linux Sparse Files

Yesterday a developer asked for a log file; using du -hs smartorder.log I saw 9.0G, compressed it to 744M, and sent it. Later the developer showed a screenshot indicating the file was actually 100 GB.

# du -hs smartorder.log
9.0G smartorder.log

# du -hs smartorder.log.tar.gz
744M smartorder.log.tar.gz

Checking with ls -l --block-size=G smartorder.log revealed the true size of 103 GB.

# ls -l --block-size=G smartorder.log
-rw-r--r-- 1 root root 103G Oct 21 09:00 smartorder.log

The discrepancy is due to Linux sparse files, which allocate disk space lazily. A sparse file initially occupies no physical blocks; space is allocated only as data is written, typically in 64 KB increments. This results in a logical size far larger than the actual disk usage.

To copy sparse files efficiently, use cp --sparse=WHEN where WHEN can be auto, always, or never. Other tools that support sparse files include tar, cpio, and rsync. For example:

# tar cSf smartorder.log.tar smartorder.log
# ls -l --block-size=G smartorder.log.tar
-rw-r--r-- 1 root root 10G Oct 21 09:57 smartorder.log.tar

To detect whether a file is sparse, the find command can report the ratio of allocated blocks to file size using the %S format:

# find ./smartorder.log -type f -printf "%S\t%p
"
0.0886597 ./smartorder.log

A value less than 1.0 indicates a sparse file. To locate all sparse files on a filesystem:

find / -type f -printf "%S\t%p
" | gawk '$1 < 1.0 {print}'

Understanding sparse files helps avoid misleading size reports and ensures proper handling during backup or transfer operations.