Why Off‑Heap Memory Can Leak on Linux—and How to Avoid It

Off‑heap memory (memory outside the JVM heap) can be allocated via DirectByteBuffer or the mmap system call, enabling zero‑copy I/O for high‑performance networking and file services.

Since the JVM garbage collector does not manage this memory, forgetting to release it or misusing it leads to persistent consumption of physical memory, eventually causing OOM crashes.

Root Causes

Code‑level mistakes: malloc without free, pointer loss, or failing to clean a DirectByteBuffer.

Resource mismanagement: file descriptors or sockets opened but not closed, mmap regions not unmapped.

Third‑party libraries that allocate off‑heap buffers and do not free them, especially after version upgrades.

Detection Methods

Use pmap -x <pid> to watch the growth of the [anon] region, read /proc/<pid>/smaps, or run a custom monitor that parses these files.

# ps -ef | grep ./server_demo
pmap -x 12345
watch -n 5 pmap -x 12345

Tools such as valgrind --leak-check=full, gperftools (HEAPPROFILE), and perf/massif can pinpoint the allocation site.

# valgrind --tool=memcheck --leak-check=full ./server_demo

Mitigation Strategies

Perform thorough code review to ensure every allocation has a matching release: every malloc / calloc has a free, every open has a close, and every mmap has a munmap. In C++ use RAII or smart pointers (e.g., std::unique_ptr<char[]>) to automate cleanup, and in Java explicitly clean DirectByteBuffer via Cleaner or reflection.

#include <stdio.h>
#include <stdlib.h>
void memory_leak_example() {
    char *ptr = (char *)malloc(1024);
    if (ptr) {
        for (int i = 0; i < 1024; i++) ptr[i] = 'a';
        /* missing free(ptr); */
    }
}
int main() { memory_leak_example(); return 0; }

Set Linux limits to bound total memory and surface leaks early.

# limit virtual memory to 512 MB
ulimit -v 524288
# limit resident memory to 512 MB
ulimit -m 524288

Combining disciplined programming, continuous monitoring, and system‑level limits preserves the performance benefits of off‑heap memory while preventing costly leaks.