Diagnosing and Resolving Native Memory Leak in Spring Boot Applications

Background: After migrating a project to the MDP framework (based on Spring Boot), the system reported unusually high swap usage. Although the JVM was configured with a 4 GB heap, physical memory consumption reached 7 GB.

Investigation Process:

1. Locate memory regions at the Java level

Enabled -XX:NativeMemoryTracking=detail and used jcmd <pid> VM.native_memory detail to view memory distribution, revealing that native memory (outside the heap) was significant.

Used pmap to identify many 64 MB address spaces not reported by jcmd , indicating native allocations.

2. Locate off‑heap memory at the system level

Applied gperftools to monitor allocations; observed malloc usage peaking at 3 GB then stabilizing around 700‑800 MB.

Ran strace -f -e brk,mmap,munmap -p <pid> but found no suspicious system calls.

Dumped memory with GDB ( gdb -p <pid> ) and inspected the binary, discovering that the Meituan Configuration Center (MCC) used Reflections to scan all JARs, which internally employed Inflater for decompression, allocating off‑heap memory.

3. Why off‑heap memory was not released

Spring Boot wrapped InflaterInputStream without explicitly freeing the native memory; it relied on the finalize method of Inflater , which may not run promptly.

Analysis of the native Inflater code showed it allocates with malloc and frees in finalize , but the glibc allocator keeps freed blocks in per‑thread memory pools (≈64 MB each), so the OS memory usage appeared unchanged.

Resolution

Configured MCC to scan only specific packages instead of all JARs, eliminating the excessive native allocations.

Upgraded to Spring Boot 2.0.5, where ZipInflaterInputStream now releases native memory explicitly, removing the reliance on GC finalization.

Custom Allocator Test

A simple malloc implementation without memory pools was built and preloaded via LD_PRELOAD to compare memory usage. The test demonstrated that native allocations remained around 700‑800 MB, while the OS reported higher usage due to mmap rounding and lazy allocation.

#include<sys/mman.h>
#include<stdlib.h>
#include<string.h>
#include<stdio.h>
void* malloc ( size_t size ) {
   long* ptr = mmap(0, size + sizeof(long), PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, 0, 0);
   if (ptr == MAP_FAILED) return NULL;
   *ptr = size;
   return (void*)(&ptr[1]);
}
void *calloc(size_t n, size_t size) { void* ptr = malloc(n*size); if (!ptr) return NULL; memset(ptr,0,n*size); return ptr; }
void *realloc(void *ptr, size_t size) { /* simplified */ }
void free(void* ptr) { if (!ptr) return; long *plen = (long*)ptr; plen--; long len = *plen; munmap((void*)plen, len + sizeof(long)); }

Conclusion: The apparent memory leak was caused by native memory allocated during JAR scanning and retained by the glibc memory pool; fixing the scan configuration or upgrading Spring Boot resolves the issue.