Boost Java Performance with Off‑Heap Caching: A Practical OHC Guide

The author introduces off‑heap caching (OHC) as a solution when a Java service’s JVM heap is well‑tuned but GC still suffers because massive local cache objects occupy most of the heap memory.

Off‑heap memory is not managed by the GC and is limited only by the physical RAM, following the relationship: physical memory = off‑heap memory + heap memory.

OHC is an open‑source library that can be used out of the box to store cache entries off‑heap.

Demo

A quick start requires adding the Maven dependency:

<dependency>
    <groupId>org.caffinitas.ohc</groupId>
    <artifactId>ohc-core</artifactId>
    <version>0.7.4</version>
</dependency>

Below is a minimal example that creates an OHCache with a custom string serializer and stores a key‑value pair:

public class OhcDemo {
    public static void main(String[] args) {
        OHCache ohCache = OHCacheBuilder.<String, String>newBuilder()
                .keySerializer(OhcDemo.stringSerializer)
                .valueSerializer(OhcDemo.stringSerializer)
                .build();
        ohCache.put("hello", "why");
        System.out.println("ohCache.get(hello) = " + ohCache.get("hello"));
    }

    public static final CacheSerializer<String> stringSerializer = new CacheSerializer<String>() {
        public void serialize(String s, ByteBuffer buf) {
            byte[] bytes = s.getBytes(Charsets.UTF_8);
            buf.put((byte) ((bytes.length >>> 8) & 0xFF));
            buf.put((byte) (bytes.length & 0xFF));
            buf.put(bytes);
        }
        public String deserialize(ByteBuffer buf) {
            int length = ((buf.get() & 0xff) << 8) + (buf.get() & 0xff);
            byte[] bytes = new byte[length];
            buf.get(bytes);
            return new String(bytes, Charsets.UTF_8);
        }
        public int serializedSize(String s) {
            byte[] bytes = s.getBytes(Charsets.UTF_8);
            if (bytes.length > 65535) throw new RuntimeException("encoded string too long: " + bytes.length + " bytes");
            return bytes.length + 2;
        }
    };
}

The cache works like a Map with put and get, but the stored objects must be serialized by the user‑provided serializer.

Comparison

A simple HashMap example that allocates 1 MB strings until the JVM heap (set to 100 MB with -Xms100m -Xmx100m) overflows demonstrates an OutOfMemoryError:

public class HashMapCacheExample {
    private static HashMap<String, String> HASHMAP = new HashMap<>();
    public static void main(String[] args) throws InterruptedException {
        TimeUnit.SECONDS.sleep(10);
        int num = 0;
        while (true) {
            String big = new String(new byte[1024 * 1024]);
            HASHMAP.put(num + "", big);
            num++;
        }
    }
}

Running the same logic with OHC (same JVM heap size) keeps the application alive for minutes because the large strings are stored off‑heap:

public class OhcCacheDemo {
    public static void main(String[] args) throws InterruptedException {
        TimeUnit.SECONDS.sleep(10);
        OHCache ohCache = OHCacheBuilder.<String, String>newBuilder()
                .keySerializer(stringSerializer)
                .valueSerializer(stringSerializer)
                .build();
        int num = 0;
        while (true) {
            String big = new String(new byte[1024 * 1024]);
            ohCache.put(num + "", big);
            num++;
        }
    }
    // stringSerializer same as in the demo above
}

Memory‑monitoring screenshots show the JVM heap filling quickly in the HashMap case, while the OHC process keeps the off‑heap memory usage stable (e.g., 6 GB configured via OHC settings) without triggering Full GC.

Source Code Overview

OHC provides two implementation strategies for off‑heap storage:

Linked: allocates off‑heap memory per entry, suitable for medium or large entries.

Chunked: pre‑allocates memory per hash segment, better for small entries but marked experimental.

The core allocation interface IAllocator defines allocate, free, and getTotalAllocated (not implemented). Managing off‑heap memory therefore boils down to these two operations.

Operating Off‑Heap Memory

Typical off‑heap allocation uses Unsafe.allocateMemory, which is analogous to C’s malloc. This bypasses ByteBuffer.allocateDirect and gives the framework full control over when memory is released, avoiding unexpected deallocation during GC.

If off‑heap memory runs out, OHC may trigger a Full GC, which can be disastrous under heavy load.

OHC defaults to a JNA‑based allocator ( JNANativeAllocator) but also provides an UnsafeAllocator that accesses sun.misc.Unsafe via reflection.

public class UnsafeAllocator implements IAllocator {
    private static final Unsafe UNSAFE = // obtain via reflection
    public long allocate(long size) { return UNSAFE.allocateMemory(size); }
    public void free(long address) { UNSAFE.freeMemory(address); }
    public long getTotalAllocated() { return 0; }
}

public class JNANativeAllocator implements IAllocator {
    public long allocate(long size) { return Native.malloc(size); }
    public void free(long address) { Native.free(address); }
    public long getTotalAllocated() { return 0; }
}

The author notes that on Linux, pre‑loading jemalloc can improve native allocation performance compared with the default

glibc

malloc

.