How I Cut Full GC Frequency from 40 to 1 in 10 Days: A JVM Tuning Journey

During a month of effort, the author optimized JVM garbage collection on a four‑node cluster (each server 2 CPU cores, 4 GB RAM) where Full GC was triggered more than 40 times per day, causing frequent server restarts.

Initial JVM startup parameters were:

-Xms1000M -Xmx1800M -Xmn350M -Xss300K -XX:+DisableExplicitGC -XX:SurvivorRatio=4 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -XX:LargePageSizeInBytes=128M -XX:+UseFastAccessorMethods -XX:+UseCMSInitiatingOccupancyOnly -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC

-Xmx1800M: maximum heap size.

-Xms1000M: initial heap size (should match -Xmx to avoid reallocation).

-Xmn350M: young generation size (recommended 3/8 of total heap).

-Xss300K: thread stack size.

First Optimization

The young generation was deemed too small, so the author increased it and aligned initial and maximum heap sizes:

-Xmn350M -> -Xmn800M<br/>-XX:SurvivorRatio=4 -> -XX:SurvivorRatio=8<br/>-Xms1000M -> -Xms1800M

After deploying the new settings on two servers for five days, Young GC frequency dropped by more than half and pause time decreased by 400 s, but Full GC frequency rose by 41 occurrences, indicating an incomplete solution.

Memory‑Leak Investigation (Second Optimization)

A leak was discovered: over 10,000 instances of object T (≈20 MB) remained in memory due to an anonymous inner‑class listener that was never released after a timeout callback.

public void doSmthing(T t){
    redis.addListener(new Listener(){
        public void onTimeout(){
            if(t.success()){
                // execute operation
            }
        }
    });
}

Removing the listener and fixing related error logs reduced the leak, but Full GC frequency remained high until the root cause was identified.

Third Optimization – Metaspace and CMS Tuning

Analysis showed that Full GC sometimes occurred even when old‑gen usage was below 30 %. The culprit was an oversized Metaspace (up to ~200 MB). The author adjusted Metaspace size and CMS occupancy thresholds, and further increased young generation size on two servers:

-Xmn350M -> -Xmn800M<br/>-Xms1000M -> -Xms1800M<br/>-XX:MetaspaceSize=200M<br/>-XX:CMSInitiatingOccupancyFraction=75

-Xmn350M -> -Xmn600M<br/>-Xms1000M -> -Xms1800M<br/>-XX:MetaspaceSize=200M<br/>-XX:CMSInitiatingOccupancyFraction=75

After ten days of testing, servers prod1 and prod2 (with larger young generation) showed significantly fewer Full GC events and higher throughput compared to prod3 and prod4.

Summary

Full GC more than once per day is abnormal.

When Full GC spikes, prioritize investigating memory leaks.

After fixing leaks, JVM tuning gains are limited; further effort may not be cost‑effective.

If CPU stays high after code checks, consult operations (e.g., cloud provider) – in this case a server issue caused 100 % CPU.

Unexpected high inbound traffic often originates from database queries; verify query conditions.

Regularly monitor GC to detect problems early.