Mastering Request Merging: Hystrix Collapser, BatchCollapser & ConcurrentHashMultiset Explained

Introduction

In typical request‑response models each request occupies its own thread and resources, causing high I/O cost when many identical operations are performed. Merging similar requests upstream before sending them downstream can greatly reduce load and improve throughput.

Hystrix Collapser

Hystrix provides a collapser that merges requests using either a custom batch method or annotations. The @HystrixCollapser annotation marks the method to be collapsed, while the corresponding @HystrixCommand defines the batch method. Important points include:

Add @HystrixCollapser on the method to be collapsed and @HystrixCommand on the batch method.

The single method accepts a single argument; for multiple arguments wrap them in a custom class.

single returns java.util.concurrent.Future<T>, batch returns java.util.List<T>, and the result count must match the input count.

Example:

public class HystrixCollapserSample {
    @HystrixCollapser(batchMethod = "batch")
    public Future<Boolean> single(String input) {
        // never executed
        return null;
    }

    public List<Boolean> batch(List<String> inputs) {
        return inputs.stream()
                     .map(it -> Boolean.TRUE)
                     .collect(Collectors.toList());
    }
}

Configuration is done via the @HystrixCollapser annotation, specifying collapserKey, batchMethod, scope (REQUEST or GLOBAL), and collapserProperties such as maxRequestsInBatch, timerDelayInMilliseconds, and requestCache.enabled.

BatchCollapser (Custom Implementation)

When the result of individual requests is not needed, a simpler collapser can be built. It stores incoming requests in a thread‑safe container (e.g., LinkedBlockingDeque) and triggers batch execution either after a time interval or when a request count threshold is reached.

Key design points:

The container must preserve order and allow safe bulk removal without explicit locking.

A ScheduledExecutorService replaces a blocking Timer to schedule periodic batch processing.

Singleton factory ensures one collapser per handler class.

public class BatchCollapser<E> implements InitializingBean {
    private volatile LinkedBlockingDeque<E> batchContainer = new LinkedBlockingDeque<>();
    private Handler<List<E>, Boolean> cleaner;
    private long interval;
    private int threshHold;

    public void submit(E event) {
        batchContainer.add(event);
        if (batchContainer.size() >= threshHold) {
            clean();
        }
    }

    private void clean() {
        List<E> transferList = new ArrayList<>(threshHold);
        batchContainer.drainTo(transferList, 100);
        if (!transferList.isEmpty()) {
            cleaner.handle(transferList);
        }
    }

    @Override
    public void afterPropertiesSet() {
        SCHEDULE_EXECUTOR.scheduleAtFixedRate(() -> {
            try {
                clean();
            } catch (Exception e) {
                logger.error("clean container exception", e);
            }
        }, 0, interval, TimeUnit.MILLISECONDS);
    }
}

ConcurrentHashMultiset

For high‑frequency duplicate data, Guava’s ConcurrentHashMultiset stores each element together with a count, allowing lock‑free increments and decrements. It is ideal for scenarios such as real‑time statistics where the same key appears many times.

if (ConcurrentHashMultiset.isEmpty()) {
    return;
}
List<Request> transferList = new ArrayList<>();
ConcurrentHashMultiset.elementSet().forEach(request -> {
    int count = ConcurrentHashMultiset.count(request);
    if (count <= 0) return;
    transferList.add(count == 1 ? request : new Request(request.getIncrement() * count));
    ConcurrentHashMultiset.remove(request, count);
});

Summary

Use Hystrix collapser when each request’s result is required despite the extra latency. Use the custom BatchCollapser for fire‑and‑forget scenarios where batching can be triggered by time or count. Use ConcurrentHashMultiset for high‑duplicate statistical workloads. Combining BatchCollapser with ConcurrentHashMultiset can further reduce downstream pressure.