Diagnosing Excessive Native Memory Usage in a Spring Boot Application Using JVM Native Memory Tracking and System Tools
After migrating a project to the MDP framework based on Spring Boot, the author observed swap overuse and physical memory far exceeding the 4 GB heap; using JVM native memory tracking, gperftools, strace, pmap, and a custom allocator, they traced a native‑memory leak to Spring Boot’s ZipInflaterInputStream during JAR scanning and resolved it by limiting scan paths.
The author migrated a project to the MDP framework (built on Spring Boot) and soon encountered frequent swap‑area warnings, with physical memory usage reaching 7 GB despite a configured 4 GB heap.
JVM start‑up parameters were set as follows:
-XX:MetaspaceSize=256M -XX:MaxMetaspaceSize=256M -XX:+AlwaysPreTouch -XX:ReservedCodeCacheSize=128m -XX:InitialCodeCacheSize=128m -Xss512k -Xmx4g -Xms4g -XX:+UseG1GC -XX:G1HeapRegionSize=4MUsing -XX:NativeMemoryTracking=detail and the command jcmd <pid> VM.native_memory detail , the author observed that the committed memory reported by the JVM was far lower than the physical memory, indicating additional native allocations outside the JVM.
Further investigation with pmap revealed many 64 MB anonymous memory regions that were not accounted for by jcmd , suggesting native‑code allocations.
System‑level tools were then employed:
gperftools to monitor malloc activity, showing a peak of ~3 GB that later dropped to 700‑800 MB.
strace (e.g., strace -f -e "brk,mmap,munmap" -p <pid> ) to trace system calls, which initially did not reveal suspicious allocations.
GDB with gdp -pid <pid> and dump memory mem.bin startAddr endAddr followed by strings mem.bin to inspect raw memory contents.
jstack to locate the thread responsible for the large mmap requests.
The investigation pinpointed the Meituan Config Center (MCC) component, which uses Reflections to scan all JARs. During JAR extraction, Spring Boot’s ZipInflaterInputStream employs the JDK Inflater , which allocates off‑heap memory via native code and relies on the finalizer to release it.
Because the finalizer runs only during GC, the off‑heap memory remained allocated, and the underlying glibc memory allocator kept the pages in its per‑thread arena, causing the OS to report a much larger resident set.
To verify the allocator’s impact, the author built a simple custom allocator (shown below) and preloaded it with LD_PRELOAD to replace glibc’s malloc:
#include
#include
#include
#include
void* malloc(size_t size) {
long* ptr = mmap(0, size + sizeof(long), PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, 0, 0);
if (ptr == MAP_FAILED) return NULL;
*ptr = size;
return (void*)(&ptr[1]);
}
void* calloc(size_t n, size_t size) {
void* ptr = malloc(n * size);
if (!ptr) return NULL;
memset(ptr, 0, n * size);
return ptr;
}
void* realloc(void* ptr, size_t size) {
if (size == 0) { free(ptr); return NULL; }
if (!ptr) return malloc(size);
long* plen = (long*)ptr; plen--;
long len = *plen;
if (size <= len) return ptr;
void* rptr = malloc(size);
if (!rptr) { free(ptr); return NULL; }
rptr = memcpy(rptr, ptr, len);
free(ptr);
return rptr;
}
void free(void* ptr) {
if (!ptr) return;
long* plen = (long*)ptr; plen--;
long len = *plen;
munmap((void*)plen, len + sizeof(long));
}Tests showed that, regardless of the allocator, the native memory usage stayed around 700‑800 MB, while the OS‑level RSS varied widely due to allocator‑specific memory‑pool behavior.
Finally, the root cause was addressed by configuring MCC to scan only specific JAR packages, and Spring Boot later added an explicit ZipInflaterInputStream implementation that releases native memory without relying on finalization. Upgrading to Spring Boot 2.0.5+ eliminates the issue.
In summary, the excessive native memory consumption was caused by off‑heap allocations performed by the Inflater during JAR scanning, combined with glibc’s arena memory‑pool retaining pages after GC; limiting the scan scope and fixing the Inflater implementation resolves the problem.
Code Ape Tech Column
Former Ant Group P8 engineer, pure technologist, sharing full‑stack Java, job interview and career advice through a column. Site: java-family.cn
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.