How to Diagnose Java Thread Deadlocks and High CPU Load with Arthas

Arthas is an online monitoring and diagnostic tool that provides a global view of application load, memory, GC, and thread status, allowing diagnosis of business issues without modifying code, such as viewing method call parameters, exceptions, execution time, and class loading information.

In this article we focus on using Arthas to analyze thread-related problems.

Deadlock Issues

Deadlocks often cause applications to hang when a thread holds a lock and other threads wait for it. Arthas offers commands to quickly locate such problems.

Using the thread command displays thread statistics; the BLOCKED count indicates blocked threads. Executing thread -b is equivalent to jstack -l | grep -i -E 'BLOCKED|deadlock', revealing the threads causing deadlocks.

After running thread -b, the output looks like:

"Thread-34" Id=93 BLOCKED on java.lang.Object@1820d623 owned by "Thread-36" Id=92
at cn.lhy.test.controller.TestController.lambda$hb$1(TestController.java:65)
- blocked on java.lang.Object@1820d623
- locked java.lang.Object@29064ce5 <---- but blocks 3 other threads!
at cn.lhy.test.controller.TestController$Lambda$1407/395258860.run(Unknown Source)
at java.lang.Thread.run(Thread.java:748)

The output directly shows the thread IDs, code locations, and the number of threads blocked, making deadlock identification clear.

High CPU Load

In production environments, high CPU usage often stems from thread activity. The thread command can be used to view thread information and stack traces.

CPU usage is a key indicator of system load. While high CPU alone may be normal, sustained high CPU leading to high load can be dangerous. Acceptable CPU usage varies with whether the system is compute‑intensive or I/O‑intensive.

To pinpoint the cause of high CPU load, first identify which threads consume CPU, such as GC threads or application threads. A dashboard can display overall process metrics, as shown below.

CPU usage per thread corresponds to the %CPU column in top -H -p, representing the ratio of incremental CPU time to the sampling interval.

If high CPU usage is caused by GC threads, consider optimizing GC (reducing Full GC frequency and duration, adjusting object allocation sizes). If caused by application code, review thread pools and code for excessive thread creation.

For detailed analysis, export a thread dump using the thread command. The following image shows the command options.

By sampling twice and comparing thread CPU times, you can calculate each thread's incremental CPU usage:

Thread CPU usage = (incremental CPU time / sampling interval) × 100%

[arthas@35]$ thread -n 3
"arthas-command-execute" Id=24 cpuUsage=70.32% deltaTime=0ms time=36ms RUNNABLE
at java.management@15-ea/sun.management.ThreadImpl.dumpThreads0(Native Method)
...
"C1 CompilerThread0" [Internal] cpuUsage=0.28% deltaTime=0ms time=1032ms
"VM Periodic Task Thread" [Internal] cpuUsage=0.07% deltaTime=0ms time=982ms
"C2 CompilerThread0" [Internal] cpuUsage=0.01% deltaTime=0ms time=1021ms
"Reference Handler" Id=2 cpuUsage=0.0% deltaTime=0ms time=2ms RUNNABLE
    at java.base@15-ea/java.lang.ref.Reference.waitForReferencePendingList(Native Method)
    at java.base@15-ea/java.lang.ref.Reference.processPendingReferences(Reference.java:241)
    at java.base@15-ea/java.lang.ref.Reference$ReferenceHandler.run(Reference.java:213)

This allows analysis of the stacks of threads with high CPU usage and identification of code issues.

Note that the statistics thread itself consumes some CPU; to reduce its impact, increase the sampling interval.