Diagnosing Java App Slowdowns: CPU Spikes, Full GC, and Deadlocks

Developers who have handled production incidents often encounter sudden system slowdowns, CPU reaching 100%, and an unusually high number of Full GC events.

Typical Root Causes

Reading a large amount of data in a single code location, exhausting memory and triggering frequent Full GC.

CPU‑intensive operations that cause the CPU to stay at high usage.

Other issues that may cause functional latency without making the whole system unavailable include blocking operations, threads stuck in WAITING state, and deadlocks caused by improper lock usage.

Full GC Overload

Full GC is the most common problem, especially after deploying new features. Two main symptoms are:

Multiple threads with CPU usage over 100% that are identified as garbage‑collection threads via jstack.

Increasing Full GC count shown by jstat -gcutil.

top - 08:31:10 up 30 min, 0 users, load average: 0.73, 0.58, 0.34
KiB Mem: 2046460 total, 1923864 used, 122596 free, 14388 buffers
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
9 root 20 0 2557160 288976 15812 S 98.0 14.1 0:42.60 java

After identifying the high‑CPU Java process (PID 9), use top -Hp 9 to list its threads and note the thread ID with the highest CPU usage. top -Hp 9 Convert the thread ID to hexadecimal for jstack lookup:

printf "%x
" 10
# output: a

Inspect the thread in the jstack dump; a VM thread (e.g., "VM Thread") indicates GC activity, while a regular application thread points to the offending code.

High CPU Due to Application Logic

If the high‑CPU thread is a user thread, the stack trace will show the specific method consuming CPU, allowing you to pinpoint the problematic line of code.

Intermittent Interface Latency

When an API occasionally takes 2–3 seconds, CPU and memory usage may appear normal. Use a load‑testing tool to increase request frequency, then collect multiple jstack logs (e.g., a1.log, a2.log) and compare them to find threads that consistently appear in the waiting state.

"http-nio-8080-exec-2" #29 daemon prio=5 os_prio=31 tid=0x00007fd08cb26000 nid=0x9603 waiting on condition
java.lang.Thread.State: TIMED_WAITING (sleeping)
    at com.aibaobei.user.controller.UserController.detail(UserController.java:18)

Threads blocked at the same line (e.g., line 18 of UserController) reveal the bottleneck.

Threads Stuck in WAITING State

Extract all TIMED_WAITING threads from a jstack dump using grep, save the output, wait a few seconds, repeat, and then compare the files to locate threads that remain in WAITING across snapshots. Those threads are likely the problematic ones.

"Thread-0" #11 prio=5 os_prio=31 tid=0x00007f9de08c7000 nid=0x5603 waiting on condition
java.lang.Thread.State: WAITING (parking)
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
    at com.aibaobei.chapter2.eg4.SyncTask.lambda$main$0(SyncTask.java:8)

Identify the user‑defined class in the stack trace to confirm the source of the wait.

Deadlock Detection

Run jstack and look for the "Found one Java-level deadlock" section. The dump lists the threads involved and the exact lines where each thread holds a lock and waits for another.

Summary

The five typical scenarios that cause system slowdown are:

Large data reads leading to memory exhaustion and frequent Full GC.

CPU‑intensive code.

Blocking operations that cause occasional latency.

Threads permanently in WAITING state.

Deadlocks.

General troubleshooting steps:

Use top to check CPU; if high, drill down with top -Hp <pid> to find the offending thread.

Convert the thread ID to hex and search the jstack dump.

If the thread is a user thread, analyze its stack trace; if it is a VM thread, monitor GC with jstat -gcutil and dump memory for analysis with Eclipse MAT.

If CPU and memory are normal, consider intermittent latency, persistent waiting threads, or deadlocks, and apply the respective analysis methods described above.