Investigation of Excessive Native Memory Usage After Migrating to Spring Boot

After migrating a project to the MDP framework (based on Spring Boot), the system began reporting excessive swap usage. Although the JVM was configured with a 4 GB heap, the physical memory consumption reached 7 GB, which was abnormal.

JVM parameters used:

-XX:MetaspaceSize=256M<br/>-XX:MaxMetaspaceSize=256M<br/>-XX:+AlwaysPreTouch<br/>-XX:ReservedCodeCacheSize=128m<br/>-XX:InitialCodeCacheSize=128m,<br/>-Xss512k -Xmx4g -Xms4g,-XX:+UseG1GC<br/>-XX:G1HeapRegionSize=4M

The actual physical memory usage is shown in the following image:

Investigation Process

1. Locate memory regions at the Java level

Added the JVM option -XX:NativeMemoryTracking=detail and restarted the application. Then executed: jcmd pid VM.native_memory detail The resulting memory distribution (image) showed that the committed memory reported by jcmd was smaller than the physical memory because it does not include native memory allocated by C code.

Using pmap revealed many 64 MB address spaces that were not listed by jcmd, suggesting native allocations.

2. Use system‑level tools to locate off‑heap memory

gperftools was employed to monitor memory allocation. The tool showed that malloc usage peaked at 3 GB and then settled around 700‑800 MB.

Since gperftools did not capture the suspicious allocations, strace was used: strace -f -e"brk,mmap,munmap" -p pid The trace did not reveal obvious allocation calls.

Next, GDB was used to dump suspicious memory regions:

gdp -pid pid<br/>dump memory mem.bin startAddress endAddress<br/>strings mem.bin

The dumped content mainly consisted of extracted JAR information, indicating that the memory was allocated during JAR decompression.

Running strace at application startup captured many 64 MB mmap allocations:

The corresponding pmap output confirmed these regions.

Finally, jstack was used to identify the thread responsible for the allocations (the thread ID obtained from strace).

jstack pid

The investigation revealed that MCC (Meituan Unified Configuration Center) used Reflections to scan all JAR packages. The scanning process relied on Spring Boot’s ZipInflaterInputStream, which allocates native memory for JAR decompression but does not release it promptly. After configuring MCC to scan only specific packages, the memory problem disappeared.

3. Why was the off‑heap memory not released?

Even though the issue was resolved, several questions remained:

Why did the previous framework not exhibit the problem?

Why was the native memory not freed?

Why were the allocations consistently 64 MB?

Why did gperftools report only ~700 MB while the OS showed much higher usage?

Investigation of Spring Boot’s source showed that it wraps InflaterInputStream with ZipInflaterInputStream. The Inflater object allocates native memory via malloc and releases it in its finalize method. Spring Boot relied on GC to trigger this finalizer, which does not guarantee immediate return of memory to the OS.

Further analysis indicated that the GNU C Library (glibc) uses per‑thread memory arenas of 64 MB on 64‑bit systems. Even after free is called, the memory often remains in the allocator’s pool rather than being returned to the OS, giving the impression of a leak.

A custom malloc library (compiled as zjbmalloc.so and preloaded with LD_PRELOAD) demonstrated that native allocations of 800 MB could result in a physical memory footprint of 1.7 GB due to page‑size rounding and lazy allocation.

gcc zjbmalloc.c -fPIC -shared -o zjbmalloc.so<br/>export LD_PRELOAD=zjbmalloc.so

These findings confirmed that the observed “leak” was largely caused by the allocator’s behavior rather than a true memory loss.

Conclusion

The memory consumption flow can be summarized as follows: MCC’s default package‑scanning configuration triggers Spring Boot’s ZipInflaterInputStream, which allocates off‑heap memory during JAR decompression. The memory is only released when the JVM’s finalizer runs, and the underlying glibc allocator retains the pages in its arena, making the OS‑level memory appear unreleased. Updating Spring Boot to version 2.0.5.RELEASE, which explicitly releases the native memory, or restricting the scan path, resolves the issue.

References

GNU C Library (glibc)

Native Memory Tracking

Spring Boot

gperftools

Btrace

Author

Ji Bing, joined Meituan in 2015, currently working on hotel C‑end services.