A Comprehensive Guide to Java Microbenchmarking with JMH

JMH Overview

In everyday development, measuring the performance of code or tools is essential, and simple repeated timing often fails due to JVM JIT compilation and interpretation mixing. Warm‑up code is therefore required to obtain stable benchmark results.

JMH (Java Microbenchmark Harness) is an official OpenJDK tool for micro‑benchmarking at the method level with microsecond precision.

Key Points for Java Benchmarking

Warm‑up before testing.

Avoid dead code elimination.

Concurrent testing.

Presenting results.

JMH Use Cases

Quantitatively analyze optimization effects of hot functions.

Measure execution time of a function and its relation to input variables.

Compare multiple implementations of a function.

Demo

1. Building the Test Project

JMH is bundled with Java 9+, but the example uses Java 8. The Maven archetype can be generated with:

$ mvn archetype:generate \
          -DinteractiveMode=false \
          -DarchetypeGroupId=org.openjdk.jmh \
          -DarchetypeArtifactId=jmh-java-benchmark-archetype \
          -DgroupId=org.sample \
          -DartifactId=test \
          -Dversion=1.0

Alternatively, add the following dependencies to an existing Maven project:

<dependency>
    <groupId>org.openjdk.jmh</groupId>
    <artifactId>jmh-core</artifactId>
    <version>${jmh.version}</version>
</dependency>
<dependency>
    <groupId>org.openjdk.jmh</groupId>
    <artifactId>jmh-generator-annprocess</artifactId>
    <version>${jmh.version}</version>
    <scope>provided</scope>
</dependency>

2. Writing the Performance Test

The example benchmarks two ways of iterating a LinkedList : by index and by foreach.

/**
 * @author Richard_yyf
 * @version 1.0 2019/8/27
 */

@State(Scope.Benchmark)
@OutputTimeUnit(TimeUnit.SECONDS)
@Threads(Threads.MAX)
public class LinkedListIterationBenchMark {
    private static final int SIZE = 10000;
    private List
list = new LinkedList<>();

    @Setup
    public void setUp() {
        for (int i = 0; i < SIZE; i++) {
            list.add(String.valueOf(i));
        }
    }

    @Benchmark
    @BenchmarkMode(Mode.Throughput)
    public void forIndexIterate() {
        for (int i = 0; i < list.size(); i++) {
            list.get(i);
            System.out.print("");
        }
    }

    @Benchmark
    @BenchmarkMode(Mode.Throughput)
    public void forEachIterate() {
        for (String s : list) {
            System.out.print("");
        }
    }
}

3. Executing the Test

JMH can run benchmarks either by building a runnable JAR:

$ mvn clean install
$ java -jar target/benchmarks.jar

or directly from a main method inside an IDE:

public static void main(String[] args) throws RunnerException {
    Options opt = new OptionsBuilder()
            .include(LinkedListIterationBenchMark.class.getSimpleName())
            .forks(1)
            .warmupIterations(2)
            .measurementIterations(2)
            .output("E:/Benchmark.log")
            .build();
    new Runner(opt).run();
}

4. Reporting Results

Sample output shows throughput (operations per second) for each method:

Benchmark                                 Mode  Cnt   Score   Error  Units
LinkedListIterationBenchMark.forEachIterate   thrpt    2 1192.380          ops/s
LinkedListIterationBenchMark.forIndexIterate  thrpt    2  206.866          ops/s

The log also contains detailed JVM information, warm‑up and measurement iterations, and thread configuration.

Annotation Reference

@BenchmarkMode

Specifies the benchmark mode (Throughput, AverageTime, SampleTime, SingleShotTime, All).

@BenchmarkMode(Mode.All)
public class LinkedListIterationBenchMark { ... }

@Benchmark
@BenchmarkMode({Mode.Throughput, Mode.SingleShotTime})
public void m() { ... }

@Warmup

Defines warm‑up iterations, e.g., iterations = 3 .

@Benchmark
@Warmup(iterations = 3)
public void m() { ... }

@Measurement

Sets the number of measurement iterations and their duration.

@Benchmark
@Measurement(iterations = 3, time = 10, timeUnit = TimeUnit.SECONDS)
public void m() { ... }

@Threads

Specifies how many threads to use during the benchmark.

@Threads(Threads.MAX)
public class LinkedListIterationBenchMark { ... }

@Fork

Number of JVM forks to execute the benchmark.

@Benchmark
@Fork(3)
public void m() { ... }

@OutputTimeUnit

Unit for displaying benchmark results (seconds, milliseconds, etc.).

@OutputTimeUnit(TimeUnit.SECONDS)
public class LinkedListIterationBenchMark { ... }

@Benchmark

Marks a method as a benchmark target, similar to JUnit’s @Test .

@Param

Defines a set of values for a field, allowing the benchmark to run with different inputs.

@Setup and @TearDown

Methods annotated with @Setup run before each benchmark iteration to prepare state; @TearDown runs after to clean up resources.

@State

Declares a class as a state holder with a specific scope (Thread, Group, Benchmark).

Running the Benchmark from Code

/**
 * Entry point for IDE execution.
 */
public static void main(String[] args) throws RunnerException {
    Options opt = new OptionsBuilder()
            .include("Helloworld")
            .exclude("Pref")
            .warmupIterations(10)
            .measurementIterations(10)
            .forks(3)
            .output("E:/Benchmark.log")
            .build();
    new Runner(opt).run();
}

Conclusion

JMH can benchmark a wide range of Java components, from logging frameworks to bean‑copy utilities. Refer to the official JMH samples for more examples.