How to Use JMH for Precise Java Microbenchmarking and Performance Optimization

Microbenchmarking Basics

Microbenchmarking focuses on measuring the performance of specific code fragments or methods, providing metrics such as throughput, execution time, CPU and memory usage to support performance optimization. Compared with full‑chain testing, it isolates a single method, enabling rapid bottleneck identification, e.g., comparing JSON versus Protobuf serialization in a tracking system.

Key Advantages

Precise Targeting : Directly test a method, eliminating external factors like network latency. Example: benchmark the UserBehavior serialization method.

Performance Comparison : Quantify and compare multiple implementations, such as different encryption algorithms.

Regression Testing : Run benchmarks after code changes to detect performance degradation.

System Stability : Continuously monitor critical methods to spot potential risks, e.g., measuring SQL query execution time.

When designing microbenchmarks, avoid common pitfalls like using System.currentTimeMillis() for timing, which lacks precision and is affected by JVM optimizations and GC. The following flawed example illustrates this issue:

long start = System.currentTimeMillis();
for (int i = 0; i < 1000000; i++) {
    // do nothing
}
long end = System.currentTimeMillis();
System.out.println("Execution time: " + (end - start) + " ms");

To achieve high‑precision, repeatable results, the Java Microbenchmark Harness (JMH) is the industry‑standard tool.

Java Microbenchmarking Tool JMH

JMH offers a simple API, extensive documentation, and community support, making it easy for Java developers and test engineers to get started. It provides high precision and repeatability through warm‑up, multiple iterations, and JVM state control, delivering nanosecond‑level accuracy.

Easy to Learn : Quick setup with Maven dependencies.

High Precision & Repeatability : Warm‑up, iterations, and controlled JVM settings eliminate JIT and GC interference.

Rich Features : Supports multithreading, GC statistics, JVM parameter configuration, and integrates with JUnit, Maven, and CI/CD pipelines.

JMH Dependency and Configuration

<dependency>
  <groupId>org.openjdk.jmh</groupId>
  <artifactId>jmh-core</artifactId>
  <version>1.33</version>
</dependency>
<dependency>
  <groupId>org.openjdk.jmh</groupId>
  <artifactId>jmh-generator-annprocess</artifactId>
  <version>1.33</version>
  <scope>provided</scope>
</dependency>

JMH Demo: Synchronized Method Performance Test

The following benchmark compares a synchronized method safe() with a non‑synchronized method notSafe() to evaluate the overhead of the synchronized keyword.

package org.funtester.performance.books.chapter11.section1;

import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.results.format.ResultFormatType;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import java.util.concurrent.TimeUnit;

@BenchmarkMode(Mode.Throughput)
@State(Scope.Thread)
@Warmup(iterations = 1, time = 1, timeUnit = TimeUnit.SECONDS)
@Measurement(iterations = 1, batchSize = 1)
@Threads(1)
@Fork(value = 3, jvmArgs = {"-Xms1G", "-Xmx1G"})
@OutputTimeUnit(TimeUnit.MICROSECONDS)
public class SyncTest {
    int i = 0;

    @Benchmark
    public void safeTest() { safe(); }

    @Benchmark
    public void notSafeTest() { notSafe(); }

    public synchronized void safe() { i++; }
    public void notSafe() { i++; }

    public static void main(String[] args) throws RunnerException {
        Options options = new OptionsBuilder()
                .include(SyncTest.class.getSimpleName())
                .result("result.json")
                .resultFormat(ResultFormatType.JSON)
                .build();
        new Runner(options).run();
    }
}

JMH Annotation Explanation : @BenchmarkMode(Mode.Throughput): Measures operations per second; other modes include AverageTime and SampleTime. @State(Scope.Thread): Provides isolated state per thread. @Warmup: Performs a warm‑up iteration to stabilize the JVM. @Measurement: Defines measurement iterations and batch size. @Threads(1): Runs a single‑threaded test. @Fork: Executes the benchmark in three separate JVM forks with 1 GB heap. @OutputTimeUnit: Outputs results in microseconds.

Run Log Example :

# Run complete. Total time: 00:00:23

Benchmark                Mode  Cnt    Score   Error  Units
SyncTest.notSafeTest    thrpt    1 2689.625          ops/us
SyncTest.safeTest       thrpt    1  366.672          ops/us

Benchmark result is saved to result.json

Result Analysis : The non‑synchronized method achieves ~2689 ops/µs, while the synchronized version drops to ~367 ops/µs, indicating a ~7× performance penalty due to lock contention. Although the absolute throughput remains high, the impact becomes significant under high concurrency, warranting multithreaded testing.

Performance Testing Applications

Serialization Performance : Compare JSON (FastJSON) versus Protobuf serialization of UserBehavior to improve data processing.

Multithread Testing : Change @Threads(8) to simulate high concurrency and assess the impact of synchronized under load.

Parameter Tuning : Increase @Warmup and @Measurement iterations (e.g., to 5) for more stable results.

Regression Testing : Re‑run JMH after code changes, such as upgrading encryption algorithms, to detect performance regressions.

By leveraging JMH, testers can pinpoint performance bottlenecks, select optimal implementations, and ensure the tracking system remains performant for millions of users.