Master Java Microbenchmarking with JMH: From Setup to Results

"If you cannot measure it, you cannot improve it".

In everyday development we often need to measure the performance of code or tools, but the JVM’s mixed JIT and interpreter execution makes it hard to obtain stable results without proper warm‑up.

The JVM continuously compiles hot code paths into machine code, which means repeated runs may yield different timings; therefore experienced developers add a warm‑up phase before measuring.

JMH (Java Microbenchmark Harness) is an official OpenJDK tool for micro‑benchmarking Java code at the method level with microsecond precision.

Key points for Java benchmarking:

Warm up before the test.

Avoid dead code elimination.

Use concurrent testing when needed.

Present results clearly.

Typical JMH use cases include quantifying optimization effects of hot methods, measuring execution time relative to input variables, and comparing multiple implementations of a function.

Demo Demonstration

The following demo shows a quick way to get started with JMH.

Test Project Build

JMH is bundled with Java 9 and later; the example uses Java 8. You can create a JMH project via Maven archetype:

$ mvn archetype:generate \
    -DinteractiveMode=false \
    -DarchetypeGroupId=org.openjdk.jmh \
    -DarchetypeArtifactId=jmh-java-benchmark-archetype \
    -DgroupId=org.sample \
    -DartifactId=test \
    -Dversion=1.0

After generation a test project appears with the standard Maven structure.

Alternatively, add the following dependencies to an existing Maven project:

<dependency>
    <groupId>org.openjdk.jmh</groupId>
    <artifactId>jmh-core</artifactId>
    <version>${jmh.version}</version>
</dependency>
<dependency>
    <groupId>org.openjdk.jmh</groupId>
    <artifactId>jmh-generator-annprocess</artifactId>
    <version>${jmh.version}</version>
    <scope>provided</scope>
</dependency>

Writing Performance Tests

The example benchmarks two ways of iterating a LinkedList: index‑based and foreach‑based.

/**
 * @author Richard_yyf
 * @version 1.0 2019/8/27
 */
@State(Scope.Benchmark)
@OutputTimeUnit(TimeUnit.SECONDS)
@Threads(Threads.MAX)
public class LinkedListIterationBenchMark {
    private static final int SIZE = 10000;
    private List<String> list = new LinkedList<>();

    @Setup
    public void setUp() {
        for (int i = 0; i < SIZE; i++) {
            list.add(String.valueOf(i));
        }
    }

    @Benchmark
    @BenchmarkMode(Mode.Throughput)
    public void forIndexIterate() {
        for (int i = 0; i < list.size(); i++) {
            list.get(i);
            System.out.print("");
        }
    }

    @Benchmark
    @BenchmarkMode(Mode.Throughput)
    public void forEachIterate() {
        for (String s : list) {
            System.out.print("");
        }
    }
}

Running the Test

JMH can run benchmarks either by building a runnable JAR or directly from a main method / unit test.

To generate a JAR:

$ mvn clean install
$ java -jar target/benchmarks.jar

For quick IDE execution, use the following main method:

public static void main(String[] args) throws RunnerException {
    Options opt = new OptionsBuilder()
        .include(LinkedListIterationBenchMark.class.getSimpleName())
        .forks(1)
        .warmupIterations(2)
        .measurementIterations(2)
        .output("E:/Benchmark.log")
        .build();
    new Runner(opt).run();
}

Result Reporting

Sample output shows throughput (operations per second) for each benchmark method:

Benchmark                                 Mode  Cnt   Score   Error  Units
LinkedListIterationBenchMark.forEachIterate thrpt   2 1192.380          ops/s
LinkedListIterationBenchMark.forIndexIterate thrpt   2  206.866          ops/s

Interpretation notes remind users to use profilers, factorial experiments, baseline and negative tests, and to involve domain experts for reliable conclusions.

Annotation Overview

Key JMH annotations explained: @BenchmarkMode: defines benchmark type (e.g., Throughput, SingleShotTime). @Warmup: number of warm‑up iterations. @Measurement: number of measurement iterations and time per iteration. @Threads: number of threads per fork. @Fork: how many separate JVM forks to run. @OutputTimeUnit: time unit for results (seconds, milliseconds, etc.). @Benchmark: marks a method as a benchmark. @Param: defines variable parameters for a benchmark. @Setup / @TearDown: code to run before/after each benchmark iteration. @State: declares a class as a shared state with a specific scope (Thread, Group, Benchmark).

Launching the Benchmark

The main method can configure inclusion/exclusion patterns, warm‑up, measurement iterations, forks, and output file:

/**
 * Only for IDE execution; command line uses build + java -jar.
 */
public static void main(String[] args) throws RunnerException {
    Options opt = new OptionsBuilder()
        .include("Helloworld")
        .exclude("Pref")
        .warmupIterations(10)
        .measurementIterations(10)
        .forks(3)
        .output("E:/Benchmark.log")
        .build();
    new Runner(opt).run();
}

Conclusion

JMH can benchmark many tools and frameworks, such as logging libraries or bean‑copy utilities. For more examples, refer to the official JMH samples repository.

https://hg.openjdk.java.net/code-tools/jmh/file/tip/jmh-samples/src/main/java/org/openjdk/jmh/samples/

The article discusses common Java performance testing pitfalls, their relation to OS and JVM internals, and demonstrates how JMH helps avoid them.