Why Is My NAS Memory Exhausted? Uncovering XFS inode and ILI Cache Culprits

While testing NAS performance with fstest, prolonged writes caused a noticeable slowdown, prompting an investigation that showed the server host’s memory usage was almost fully consumed.

1. Check memory usage with top :

# top -M

top - 14:43:12 up 14 days, 6 min, 1 user, load average: 8.36, 8.38, 8.41

Mem: 63.050G total, 62.639G used, 420.973M free, 33.973M buffers

The %MEM column showed only tiny values for the top‑ranking processes, yet memory was nearly exhausted.

2. Inspect kernel memory with vmstat -m :

# vmstat -m

Cache                       Num  Total   Size  Pages

xfs_ili                  7564110 8351947    224     17

xfs_inode                7564205 8484180   1024      4

These two entries were consuming a large amount of memory.

3. Examine slab caches with slabtop -s c | head :

# slabtop -s c | head

Active / Total Objects (% used)    : 31807723 / 35664583 (89.2%)

Active / Total Slabs (% used)      : 3259180 / 3259251 (100.0%)

Active / Total Size (% used)       : 11242773.43K / 12756788.05K (88.1%)

8480676 7565420  89%    1.00K 2120169        4  xfs_inode

8351794 7565375  90%    0.22K 491282       17  xfs_ili

The xfs_inode cache used about 8 GB and xfs_ili about 2 GB.

4. What are xfs_ili and xfs_inode ? They are kernel structures for XFS filesystem logging and inode metadata. Sample kernel definitions:

xfs_inode_zone = kmem_zone_init_flags(sizeof(xfs_inode_t), "xfs_inode", KM_ZONE_HWALIGN | KM_ZONE_RECLAIM | KM_ZONE_SPREAD, xfs_fs_inode_init_once);

xfs_ili_zone = kmem_zone_init_flags(sizeof(xfs_inode_log_item_t), "xfs_ili", KM_ZONE_SPREAD, NULL);

Thus they represent the inode cache and the inode‑log‑item cache.

5. Estimate memory usage: The NFS server hosts 14 volumes, each formatted with an XFS journal size of 128 MiB. 14 × 128 MiB ≈ 1.96 GiB, matching the xfs_ili consumption. Together, xfs_inode and xfs_ili account for roughly 10 GiB, leaving about 50 GiB of memory used by other filesystem caches.

6. Release memory manually:

# sync

# echo 3 > /proc/sys/vm/drop_caches

Running these commands freed a substantial amount of RAM.

7. Conclusion: Limited memory can degrade NFS performance, especially as the number of XFS‑based volumes grows. Proper memory provisioning on the NFS server is essential for stable performance.