Understanding Filesystem Cache and Log Rotation with top, lsof, and Compression

We often use the top command, whose header shows a line such as cached indicating how much memory the operating system has used for filesystem cache.

Filesystem cache stores frequently accessed parts of files in RAM, making access at least a hundred times faster than reading from disk.

While caching useful files is beneficial, large log files can also occupy cache space without contributing to production performance.

A useful command to identify what occupies the cache is:

/usr/sbin/lsof | sort -k 7 -rn | less

This lists open files sorted by size, effectively showing which files are cached.

Typical output may show many database files, e.g.:

postgres 28621 postgres 151u REG 8,7 1073741824 27265770 laser/export/pg920_data/base/823620/2990054

However, you might also see a massive log file such as:

httpd 10771 laser 30u REG 8,17 21474918912 11 /var/log/httpd/access.log

Because log files are useful for analysis but not for real‑time production, they should be kept small to avoid unnecessary cache consumption.

The logical conclusion is to rotate logs frequently (hourly) and compress them; compression creates a new file and deletes the original, prompting the OS to drop the old data from cache.

Compressing logs also reduces the load caused by rotating large files, which is especially important for globally distributed systems.

To view compressed logs without manual decompression, you can use:

zless (equivalent to gzip -cd xxx | less ) and zcat (equivalent to gzip -cd xxx | cat ).

Author: He Weiping, search and database researcher at Qunar, former Yahoo China search engineer, and translator of PostgreSQL and Programming Perl Chinese editions.