Master JVM Performance: Memory Model, OOM Cases, and Tuning Tools

Java Memory Model Review

We start by revisiting the HotSpot JVM memory model, which is divided into three parts: the class loader, the runtime data areas, and the heap.

Class Loader

The class loader loads compiled .class files and stores class metadata in the method area.

Runtime Data Area

Thread stacks, native method stacks, and the program counter are thread‑private. The heap stores objects created at runtime. Since Java 8 the permanent generation has been replaced by Metaspace, which resides in native memory.

G1’s heap allocation strategy is shown below (Java 8 example, G1 and ZGC are not discussed further).

Execution Engine

The interpreter executes bytecode line‑by‑line, while the JIT compiler translates hot code paths into native machine code.

When to Perform JVM Tuning

Symptoms such as slow response, service unavailability, low throughput, excessive memory consumption, frequent GC pauses, or OutOfMemoryError indicate the need for tuning.

Key Metrics

Application Memory Usage – controlled by -Xms (initial heap) and -Xmx (maximum heap).

Throughput – e.g., transactions per second or batch jobs completed per hour.

Response Latency – time from request receipt to response delivery.

Common OOM Exceptions and Reproduction

java.lang.OutOfMemoryError: Java heap space

Caused by memory leaks or insufficient heap size. Example code that leaks memory:

public class HeapSize {<br/>    public static void main(String[] args){<br/>        List<User> list = new ArrayList<>();<br/>        User user = new User();<br/>        while (true){<br/>            list.add(user);<br/>        }<br/>    }<br/>}

Running with -Xmx512m eventually throws:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space<br/>at java.util.ArrayList.grow(ArrayList.java:261)<br/>...

java.lang.OutOfMemoryError: GC overhead limit exceeded

This occurs when GC consumes >98% CPU time but reclaims <2% heap. Example:

public class GcOverhead {<br/>    public static void main(String[] args){<br/>        Map map = System.getProperties();<br/>        Random r = new Random();<br/>        while (true) {<br/>            map.put(r.nextInt(), "value");<br/>        }<br/>    }<br/>}

Run with -Xmx45m -XX:+UseParallelGC -XX:+PrintGCDetails to reproduce the error.

java.lang.OutOfMemoryError: Requested array size exceeds VM limit

int[] arr = new int[Integer.MAX_VALUE - 1];

Another example shows the error after a Java heap space OOM.

for (int i = 3; i >= 0; i--) {<br/>    try {<br/>        int[] arr = new int[Integer.MAX_VALUE-i];<br/>        System.out.format("Successfully initialized an array with %,d elements.
", Integer.MAX_VALUE-i);<br/>    } catch (Throwable t) {<br/>        t.printStackTrace();<br/>    }<br/>}

java.lang.OutOfMemoryError: Metaspace

Increase Metaspace with -XX:MaxMetaspaceSize=2m to avoid the error.

java.lang.OutOfMemoryError: Out of swap space

Occurs when the JVM’s total memory demand exceeds the host’s physical + swap memory. Adjusting ulimit or disabling swap is recommended.

java.lang.OutOfMemoryError: Unable to create native threads

Caused by insufficient memory per thread ( -Xss) or OS limits ( ulimit -u, /proc/sys/kernel/threads-max, /proc/sys/kernel/pid_max).

private static void createSlowThread(){<br/>    try {<br/>        System.out.println(Thread.currentThread());<br/>        Thread.sleep(15000);<br/>    } catch (InterruptedException e){<br/>        e.printStackTrace();<br/>    }<br/>}<br/>public static void test1(){<br/>    while (true){<br/>        new Thread(() -> createSlowThread()).start();<br/>    }<br/>}

JVM Built‑in Monitoring Tools

JPS

Lists all JVM processes. Example:

jps -mlvV

jstat

Shows JVM performance data. Example:

jstat -gc -h 2 44074 1s 5

jmap

Displays object or heap information. Example: jmap -dump:live,format=b,file=filename.bin or

jmap -histo:live 44074

jinfo

Shows or modifies JVM parameters. Example: jinfo 44074 or

jinfo -flag +HeapDumpAfterFullGC 44074

jstack

Prints thread stack traces and lock information. Example:

jstack 44074

Common JVM Tuning Parameters

Heap size: -Xmx4g, -Xms2g, -Xmn1g (young generation ~3/8 of total heap).

Thread stack: -Xss512k.

Metaspace: -XX:MetaspaceSize=512m, -XX:MaxMetaspaceSize=512m.

GC selection: -XX:+UseSerialGC, -XX:+UseParallelGC, -XX:+UseConcMarkSweepGC, etc.

GC logging: -XX:+PrintGC, -XX:+PrintGCDetails, -XX:+PrintGCTimeStamps, -Xloggc:filename.

Third‑Party Monitoring Tools

Eclipse MAT

Download from https://www.eclipse.org/mat/downloads.php. Used to analyze heap dumps and detect memory leaks.

java -jar -XX:+PrintGC -XX:+PrintGCDetails -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=./ spring-boot-mybatis-1.0-SNAPSHOT.jar

Arthas (Alibaba)

Diagnostic tool for online debugging, class location, and real‑time JVM monitoring. URLs:

https://alibaba.github.io/arthas/arthas-tutorials?language=cn&id=arthas-basics

and ...&id=arthas-advanced.

IBM Heap Analyzer

Graphical tool for heap leak detection (no longer maintained; MAT is recommended). URL: https://www.ibm.com/support/pages/ibm-heapanalyzer.

Tuning Case Studies

Deadlock Diagnosis

public static void test() {<br/>    Object lockA = new Object();<br/>    Object lockB = new Object();<br/>    new Thread(() ->{<br/>        synchronized (lockA){<br/>            Thread.sleep(2000);<br/>            synchronized (lockB){<br/>                System.out.println("thread 1");<br/>            }<br/>        }<br/>    }).start();<br/>    new Thread(() ->{<br/>        synchronized (lockB){<br/>            synchronized (lockA){<br/>                System.out.println("thread 2");<br/>            }<br/>        }<br/>    }).start();<br/>}

Running jstack reveals the blocked threads.

Heap Parameter Settings

Adding -XX:+PrintGC and -XX:+PrintGCDetails produces detailed GC logs such as:

[Full GC (Allocation Failure) [PSYoungGen: 0K->0K(150528K)] [ParOldGen: 243998K->142439K(172032K)] 243998K->142439K(322560K), [Metaspace: 47754K->47754K(1093632K)], 3.6879500 secs] [Times: user=3.91 sys=0.00, real=3.69 secs]

Analysis of the numbers guides heap size, young generation ratio, and GC pause time tuning.

Heap Size Calculation

Based on observed old‑generation usage (≈139 MiB), set -Xms and -Xmx to 3–4 × 139 MiB and allocate the young generation as 3/8 of the total heap.

Metaspace Size Calculation

Observed metaspace usage (≈47 MiB) suggests setting -XX:MetaspaceSize and -XX:MaxMetaspaceSize to 1.2–1.5 × 47 MiB.

java -jar -Xms556m -Xmx556m -Xmn208m -XX:MetaspaceSize=70m -XX:MaxMetaspaceSize=70m -XX:+PrintGC -XX:+PrintGCDetails -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=./ spring-boot-mybatis-1.0-SNAPSHOT.jar