Mastering JVM Thread Dumps: From Diagnosis to Kubernetes Automation

What is a JVM thread dump?

A JVM thread dump is a snapshot of the state of all Java threads at a specific moment. It records each thread’s call stack, current status (running, waiting, blocked), and lock information, providing a detailed view of thread activity for debugging and performance analysis.

Why thread dumps matter in fault testing

In chaos engineering and stability testing, engineers deliberately inject failures (e.g., network latency, service crashes, resource bottlenecks) to evaluate system resilience. Thread dumps capture the exact thread state during these scenarios, helping locate deadlocks, resource leaks, busy‑wait loops, and other performance problems.

Typical use cases

Detecting deadlocks and blocking operations : Analyzing stack traces can reveal circular lock dependencies or threads waiting on external resources such as database queries.

Identifying abnormal thread‑count growth : Thread dumps expose orphaned or leaked threads caused by mis‑configured thread pools or unfinished asynchronous tasks.

Finding CPU‑intensive busy‑wait loops : Stack traces show tight loops or heavy computation that drive CPU usage, allowing algorithmic optimizations.

Understanding load and concurrency behavior : The distribution of thread states at a point in time informs thread‑pool sizing and task‑scheduling decisions.

Common ways to capture thread dumps

1. Using jstack command

jstack <pid> > thread_dump.txt

Identify the Java process ID with jps. In Kubernetes, run the command inside the container via kubectl exec. Frequent use may introduce a small performance impact, so prefer low‑traffic periods.

2. Sending kill -3 (SIGQUIT)

kill -3 <pid>

The JVM prints a thread dump to its standard output or configured log file (e.g., catalina.out). Ensure the output is not redirected to /dev/null or an inaccessible file.

3. Using JConsole or VisualVM

Connect to the Java process via JMX, open the “Threads” tab, and export a thread dump. These GUI tools are useful for interactive analysis but require proper JMX configuration and network permissions.

4. Programmatic call Thread.getAllStackTraces()

// Capture stack traces of all live threads
Map<Thread, StackTraceElement[]> traces = Thread.getAllStackTraces();
for (Map.Entry<Thread, StackTraceElement[]> entry : traces.entrySet()) {
    Thread thread = entry.getKey();
    StackTraceElement[] stack = entry.getValue();
    System.out.println("Thread: " + thread.getName());
    for (StackTraceElement element : stack) {
        System.out.println("\t" + element);
    }
}

This method can be embedded in test frameworks or chaos experiments. Limit invocation frequency in high‑concurrency environments to avoid performance degradation.

5. Kubernetes environment

kubectl exec -it <pod-name> -- jstack <pid> > /tmp/thread_dump.txt

Because containers are dynamic, manual execution is inefficient. Automate collection with scripts or monitoring tools (e.g., Prometheus + Grafana) that trigger a dump when CPU usage exceeds a threshold and upload the file to a log system.

Automating thread dumps with Fabric8

The Fabric8 Java client simplifies remote operations on Kubernetes clusters. It can locate pods, select containers, execute shell commands, and capture output, enabling fully automated thread‑dump workflows.

import io.fabric8.kubernetes.client.DefaultKubernetesClient;
import io.fabric8.kubernetes.client.KubernetesClient;
import io.fabric8.kubernetes.client.dsl.ExecWatch;

KubernetesClient client = new DefaultKubernetesClient();
String namespace = "default"; // target namespace
String podName = "my-app-pod"; // pod identifier
String containerName = "my-app-container"; // container inside the pod
String command = "jstack $(pgrep java) > /tmp/thread_dump.txt";

// Execute jstack inside the container
ExecWatch exec = client.pods()
    .inNamespace(namespace)
    .withName(podName)
    .inContainer(containerName)
    .writingOutput(System.out)
    .exec("sh", "-c", command);
Thread.sleep(1000); // wait for command to finish

// Optionally read the generated file
ExecWatch cat = client.pods()
    .inNamespace(namespace)
    .withName(podName)
    .inContainer(containerName)
    .writingOutput(System.out)
    .exec("sh", "-c", "cat /tmp/thread_dump.txt");

client.close();

This approach allows test engineers to capture thread dumps without manual container login, integrate the process into CI/CD pipelines, and accelerate root‑cause analysis in both development and production environments.