Improving High‑Concurrency System Performance with Coroutines and the Quasar Framework

Background

JD.com’s phone‑voucher system experiences highly uneven request rates, causing a sudden surge of concurrent requests during peak periods. To handle the load, the service spawns many threads, leading to high context‑switch overhead, thread‑waiting, and overall performance degradation, while machine utilization remains low.

System Load Issues

Large numbers of threads increase context switches and waiting time, raising system load.

Low CPU Utilization

Although many threads are created, most spend most of their time blocked on I/O, so CPU remains under‑utilized.

Root Cause Analysis

The workload is I/O‑bound; improving CPU usage therefore requires a better I/O model.

I/O Model Overview

Blocking (Synchronous) I/O : The caller blocks until the kernel finishes the operation.

Non‑blocking (Synchronous) I/O : The call returns immediately with a status; the caller must poll until data is available.

I/O Multiplexing : Uses select (or similar) to monitor multiple descriptors, reducing polling overhead but still involves blocking kernel calls.

Asynchronous I/O : The kernel notifies the application via callbacks (Proactor pattern) when data is ready, eliminating user‑space polling.

Technical Selection – Coroutines

Coroutines provide asynchronous behavior with a synchronous coding style, improving maintainability and scalability while keeping hardware utilization high.

What Is a Coroutine?

A coroutine is a lightweight user‑level construct that yields control on blocking operations, records its stack state, and resumes later without kernel‑mode context switches.

Coroutine Frameworks

JVM does not natively support coroutines; third‑party libraries such as Quasar (via bytecode instrumentation) enable them.

Quasar Architecture

Quasar uses Java instrumentation (ASM or Javassist) to weave bytecode, adding state machines to methods annotated with @Suspendable or SuspendExecution . The framework provides a Fiber abstraction, a FiberForkJoinScheduler , and related classes to manage execution.

public interface Instrumentation {
    // class file transformer
    void addTransformer(ClassFileTransformer transformer);
    // retransform classes at runtime
    void retransformClasses(Class... classes) throws UnmodifiableClassException;
}

public Future
retransformClasses(Class... classes) throws UnmodifiableClassException {
    // ... implementation ...
}

Comparison with Threads

Threads are scheduled by the OS and involve kernel‑mode switches; coroutines are scheduled by the application, incur only user‑mode switches, use far less stack memory (≈1 KB vs. 1 MB), and avoid blocking the underlying thread during I/O.

Thread‑Pool Drawbacks

Thread pools suffer from fake throughput (rejected requests counted as throughput), queue waiting delays, and blocking on I/O, which coroutines can eliminate.

Project Implementation

1. Adding Quasar Dependency

<dependency>
    <groupId>co.paralleluniverse</groupId>
    <artifactId>quasar-core</artifactId>
    <version>0.7.9</version>
    <classifier>jdk8</classifier>
</dependency>

2. Declaring Suspendable Methods

<plugin>
    <groupId>com.vlkan</groupId>
    <artifactId>quasar-maven-plugin</artifactId>
    <version>0.7.3</version>
    <configuration>
        <check>true</check>
        <debug>true</debug>
        <verbose>true</verbose>
    </configuration>
    <executions>
        <execution>
            <phase>compile</phase>
            <goals>
                <goal>instrument</goal>
            </goals>
        </execution>
    </executions>
</plugin>

3. Simple Coroutine Example

Integer result = FiberUtil.runInFiber(new SuspendableCallable
() {
    @Override
    public Integer run() throws SuspendExecution, InterruptedException {
        LOGGER.info("run in fiber begin");
        Fiber.sleep(100);
        LOGGER.info("run in fiber end");
        return 1;
    }
});

Fiber
fiber1 = new Fiber<>("fiber_1", new SuspendableCallable
() {
    @Override
    public String run() throws SuspendExecution, InterruptedException {
        LOGGER.info("fiber_1 begin");
        String str = HttpClient.doGet("http://qa-configcenter.jd.com/");
        LOGGER.info("fiber_1 end");
        return "1";
    }
}).start();

4. Replacing ThreadPool with FiberPool

Future future = ThreadPool().submit(new MarkTask());
return future.get(100, TimeUnit.MILLISECONDS);

becomes

Future future = FiberPool.submit(new MarkTask(request));
return future.get(100, TimeUnit.MILLISECONDS);

public class FiberPool implements ExecutorService {
    @Override
    public
Future
submit(Callable
task) {
        Fiber
fiber = new Fiber<>("FIBER_POOL", new SuspendableCallable
() {
            @Override
            public T run() throws SuspendExecution, InterruptedException {
                try { return task.call(); }
                catch (SuspendExecution e) { logger.error("FiberPool SuspendExecution", e); throw e; }
                catch (Exception e) { logger.error("FiberPool exception", e); }
                return null;
            }
        }).start();
        return translate(fiber);
    }
    // translate method omitted for brevity
}

Results

After introducing coroutines, request throughput increased noticeably, queue‑waiting time disappeared, and CPU load dropped because thread‑context switches were eliminated.

Risk & Caveats

Do not place suspendable calls inside synchronized blocks; doing so can cause lock inconsistencies when a coroutine yields and resumes on a different thread. Also, Quasar requires explicit SuspendExecution declarations; missing annotations lead to runtime errors such as repeated execution or NullPointerExceptions.

Conclusion

Using coroutine‑based asynchronous programming (via Quasar) allows developers to write synchronous‑style code while achieving the performance of asynchronous I/O, improving scalability and maintainability for I/O‑bound Java services.