Why Linux Cache Isn’t Always Free Memory: Deep Dive into Buffers, tmpfs, and mmap

In Linux systems the free command is commonly used to view memory usage. On a RHEL6 system the output shows total, used, free, shared, buffers, and cached values, with the default unit being kilobytes.

What is buffer/cache?

In Linux memory management, buffer cache refers to the cache for block‑device I/O, while page cache is the cache for file data. Historically buffers cached writes to block devices and page cache cached reads, but modern kernels treat them separately: page cache handles memory pages, and buffer cache handles block‑device buffers.

What is page cache?

Page cache stores file data to accelerate read/write operations. It is also used by mmap and other mechanisms that map files into memory.

What is buffer cache?

Buffer cache caches data for block‑device operations, such as during filesystem formatting, allowing the kernel to write back only dirty parts of a page.

How is cache reclaimed?

The kernel frees memory when it is low, primarily by dropping buffer and page caches. However, clearing caches incurs I/O cost because dirty data must be written back before release.

Cache can be manually cleared by writing to /proc/sys/vm/drop_caches:

# echo 1 > /proc/sys/vm/drop_caches   # free pagecache only
# echo 2 > /proc/sys/vm/drop_caches   # free slab objects (dentries, inodes)
# echo 3 > /proc/sys/vm/drop_caches   # free both pagecache and slab

Can all cache be reclaimed?

Not all cache is releasable. For example, files stored in tmpfs occupy page cache and remain until the files are deleted. Creating a 20 GB tmpfs and writing a 13 GB file shows the cached value increasing by 13 GB, and even after dropping caches the memory is still used until the file is removed.

Shared memory

POSIX shared memory (created with shmget) also uses tmpfs under the hood, so its pages stay in cache until the segment is removed with shmctl(..., IPC_RMID) or ipcrm. A test program allocating ~2 GB of shared memory demonstrates that the cached memory grows and is not freed by drop_caches until the segment is deleted.

mmap

Memory mapped with mmap using the MAP_SHARED flag behaves similarly: the kernel stores the pages in cache, and they remain until the mapping is unmapped with munmap. A 2 GB mmap test shows the cached value increasing by 2 GB and only returning to normal after the process exits and the mapping is released.

Key takeaways

Releasing cache can cause a spike in I/O because the data must be written back.

Files in tmpfs occupy cache and are not freed until the files are deleted.

Shared memory created with shmget occupies cache and is only released when the segment is removed.

Memory obtained via mmap with MAP_SHARED occupies cache and is released only after munmap or process termination.

Both shmget and MAP_SHARED mmap ultimately use tmpfs, so their memory is accounted as cache.

Understanding these details helps interpret the free output beyond the superficial “used vs. free” view and assess whether memory usage is truly problematic.