Request Collapsing Techniques: Hystrix Collapser, Custom BatchCollapser, and ConcurrentHashMultiset

Introduction

Combining similar or duplicate requests in the upstream system before sending them downstream can significantly lower the load on downstream services and increase overall system throughput. The article introduces three request‑combining techniques— hystrix collapser , ConcurrentHashMultiset , and a custom BatchCollapser —and compares their implementations and applicable scenarios.

Hystrix Collapser

Overview

Hystrix, an open‑source library from Netflix, provides a request‑collapsing feature called HystrixCollapser . It works together with the circuit‑breaker to keep web servers stable under high concurrency. The collapser can be used via annotations in the hystrix‑javanica module, requiring the hystrix‑core and hystrix‑javanica dependencies and an explicit HystrixAspect bean configuration.

<aop:aspectj-autoproxy/>
<bean id="hystrixAspect" class="com.netflix.hystrix.contrib.javanica.aop.aspectj.HystrixCommandAspect"/>

Usage

Add @HystrixCollapser on the method that needs to be merged and @HystrixCommand on the batch method.

The single method accepts a single argument; for multiple arguments wrap them in a custom class.

The single method returns java.util.concurrent.Future<SingleReturn> , while the batch method returns java.util.List<SingleReturn> with the same order as the input list.

Example

public class HystrixCollapserSample {
    @HystrixCollapser(batchMethod = "batch")
    public Future
single(String input) {
        return null; // single method is not executed directly
    }

    public List
batch(List
inputs) {
        return inputs.stream().map(it -> Boolean.TRUE).collect(Collectors.toList());
    }
}

Implementation Details

When a method annotated with @HystrixCollapser is invoked, Spring creates a collapser instance (or reuses an existing one) and stores the request arguments in a ConcurrentHashMap<RequestArgumentType, CollapsedRequest> .

A timer thread periodically drains the stored requests, builds a batch request, executes the batch method, and maps the results back to the original futures.

Note: because the collapser waits for the timer to fire, each request incurs an additional latency of roughly timerInterval/2 ms.

BatchCollapser (Custom Implementation)

Design

The custom BatchCollapser focuses on time‑based and count‑based merging without caring about individual request results. Business threads submit requests to a container; when the container reaches a threshold or a timer expires, the accumulated requests are sent downstream in a single batch.

Key Points

Uses LinkedBlockingDeque from java.util.concurrent as a thread‑safe container.

Provides a submit(E event) method that adds the event to the deque and triggers immediate merging if the size exceeds a configured threshold.

A single‑threaded ScheduledExecutorService runs the clean() method at a fixed interval to drain the deque and invoke a user‑provided Handler<List<E>, Boolean> .

public class BatchCollapser
implements InitializingBean {
    private static final ScheduledExecutorService SCHEDULE_EXECUTOR = Executors.newScheduledThreadPool(1);
    private volatile LinkedBlockingDeque
batchContainer = new LinkedBlockingDeque<>();
    private Handler
, Boolean> cleaner;
    private long interval;
    private int threshHold;
    // ... constructor, afterPropertiesSet(), submit(), clean() ...
}

Configuration

The collapser is obtained via a static factory method that ensures a singleton per handler class using a double‑checked locking pattern.

public static
BatchCollapser
getInstance(Handler
, Boolean> cleaner, int threshHold, long interval) {
    // double‑checked locking to create a singleton per handler class
}

ConcurrentHashMultiset

Design

Guava’s ConcurrentHashMultiset stores elements together with a count, allowing duplicate insertions to increment the count instead of overwriting. It is lock‑free and suitable for high‑frequency counting scenarios.

Implementation Example

if (ConcurrentHashMultiset.isEmpty()) {
    return;
}
List
transferList = Lists.newArrayList();
ConcurrentHashMultiset.elementSet().forEach(request -> {
    int count = ConcurrentHashMultiset.count(request);
    if (count <= 0) return;
    transferList.add(count == 1 ? request : new Request(request.getIncrement() * count));
    ConcurrentHashMultiset.remove(request, count);
});

Summary of Scenarios

hystrix collapser : use when each request’s result is needed and the extra latency is acceptable.

BatchCollapser : use when the result is not required and merging can be triggered by time or count thresholds.

ConcurrentHashMultiset : ideal for high‑frequency statistical aggregation where duplicate elements need counting.

By combining these techniques—e.g., using ConcurrentHashMultiset as the container inside a BatchCollapser —developers can achieve both time‑based and count‑based merging while benefiting from lock‑free counting.