Request Merging Techniques: Hystrix Collapser, BatchCollapser, and ConcurrentHashMultiset

In many services a request‑response model creates a separate thread and memory space for each call, which can cause heavy I/O overhead when many identical requests are processed. Merging similar or duplicate requests upstream before forwarding them downstream can dramatically reduce downstream load and increase overall throughput.

# Preface

During my spare time I investigated request merging, compared several libraries, translated Hystrix Javanica source code, and implemented a simple custom collapser. The problem is a bit niche, so I share my findings here.

# 1. Hystrix Collapser

Hystrix, an open‑source library from Netflix, provides a circuit breaker and a request collapser. The collapser can be configured via annotations ( @HystrixCollapser and @HystrixCommand ) or XML. Typical usage requires adding two dependencies: hystrix-core and hystrix-javanica , and enabling the HystrixAspect bean in Spring.

<aop:aspectj-autoproxy/>
<bean id="hystrixAspect" class="com.netflix.hystrix.contrib.javanica.aop.aspectj.HystrixCommandAspect"/>

The collapser works by annotating the single request method with @HystrixCollapser(batchMethod="batch") and the batch method with @HystrixCommand . Important points:

Annotate the method to be collapsed with @HystrixCollapser and the batch method with @HystrixCommand .

The single method can only accept one parameter; for multiple parameters wrap them in a custom object. The batch method receives a List<SingleParam> .

The single method returns Future<SingleReturn> , the batch method returns List<SingleReturn> , and the result list order must match the input order.

Implementation steps (simplified):

public class HystrixCollapserSample {
    @HystrixCollapser(batchMethod = "batch")
    public Future
single(String input) {
        return null; // never executed directly
    }
    public List
batch(List
inputs) {
        return inputs.stream().map(it -> Boolean.TRUE).collect(Collectors.toList());
    }
}

Internally, Hystrix registers an aspect bean, detects the annotation, creates a collapser instance, stores requests in a concurrent map, and uses a timer thread to trigger batch execution. The timer adds latency (approximately half the timer interval) to each request.

# Configuration

Configuration is done on the @HystrixCollapser annotation and includes both specific and generic Hystrix command properties:

collapserKey (optional, defaults to method name)

batchMethod (required, name of the batch method)

scope (REQUEST or GLOBAL; REQUEST creates a collapser per request, often not useful)

collapserProperties for generic command settings such as maxRequestsInBatch , timerDelayInMilliseconds , and requestCache.enabled

@HystrixCollapser(
    batchMethod = "batch",
    collapserKey = "single",
    scope = com.netflix.hystrix.HystrixCollapser.Scope.GLOBAL,
    collapserProperties = {
        @HystrixProperty(name = "maxRequestsInBatch", value = "100"),
        @HystrixProperty(name = "timerDelayInMilliseconds", value = "1000"),
        @HystrixProperty(name = "requestCache.enabled", value = "true")
    })

# 2. BatchCollapser (Custom Implementation)

When the result of each request is not needed, a lightweight custom collapser can be built. The business thread simply adds request parameters to a container; when the container reaches a size threshold or a timer fires, the accumulated requests are sent downstream in a single batch.

Design highlights:

A thread‑safe container (e.g., LinkedBlockingDeque ) stores incoming requests.

A scheduled timer thread periodically drains the container and invokes a user‑provided handler.

When the container size exceeds a configurable threshold, the batch is triggered immediately in the business thread.

public class BatchCollapser
implements InitializingBean {
    private static final Logger logger = LoggerFactory.getLogger(BatchCollapser.class);
    private static final ScheduledExecutorService SCHEDULE_EXECUTOR = Executors.newScheduledThreadPool(1);
    private volatile LinkedBlockingDeque
batchContainer = new LinkedBlockingDeque<>();
    private Handler
, Boolean> cleaner;
    private long interval;
    private int threshHold;
    // constructor, afterPropertiesSet (schedules timer), submit(E), clean(), getInstance(...)
}

The implementation uses double‑checked locking to maintain a singleton per handler class, ensuring global reuse while keeping the container thread‑safe.

# 3. ConcurrentHashMultiset

Guava provides ConcurrentHashMultiset , a thread‑safe multiset that counts occurrences of each element instead of discarding duplicates. It is ideal for high‑frequency counting scenarios (e.g., statistics, rate limiting) because it aggregates duplicate requests in memory before a single downstream operation.

if (ConcurrentHashMultiset.isEmpty()) {
    return;
}
List
transferList = Lists.newArrayList();
ConcurrentHashMultiset.elementSet().forEach(request -> {
    int count = ConcurrentHashMultiset.count(request);
    if (count <= 0) return;
    transferList.add(count == 1 ? request : new Request(request.getIncrement() * count));
    ConcurrentHashMultiset.remove(request, count);
});

# Summary

Choosing the right merging technique depends on the requirements:

Hystrix Collapser : need individual results; acceptable extra latency per request.

BatchCollapser : results are not needed; merging can be triggered by time or count thresholds.

ConcurrentHashMultiset : high duplicate‑rate counting scenarios, where aggregating counts reduces downstream pressure.

In practice, a custom BatchCollapser can internally use a ConcurrentHashMultiset as its container to combine the benefits of both approaches.

Note: The article also contains promotional messages encouraging readers to reply to the public account for gifts, join groups, and view related technical articles.