Boost SpringBoot API Throughput 10× with a Simple Async Conversion

Preface

Servlet 3.0 before: each HTTP request is handled by a thread from start to finish.

Servlet 3.0 after: provides async processing, allowing the container to release the thread and related resources, reducing load and increasing service throughput.

SpringBoot offers four ways to implement async endpoints (ResponseBodyEmitter, SseEmitter, StreamingResponseBody are not covered here): AsyncContext, Callable, WebAsyncTask, DeferredResult.

Note: Server‑side async is invisible to the client; the result type does not change. For a single request, async may increase response time slightly.

Implementation based on Callable

In a controller, returning a java.util.concurrent.Callable signals an async endpoint.

@GetMapping("/testCallAble")
public Callable<String> testCallAble() {
    return () -> {
        Thread.sleep(40000);
        return "hello";
    };
}

Processing steps:

Controller returns a Callable.

Spring MVC calls request.startAsync() and submits the Callable to an AsyncTaskExecutor for execution in a separate thread.

DispatcherServlet and all filters exit the servlet thread while the response remains open.

When the Callable produces a result, Spring MVC dispatches the request back to the servlet container to finish processing.

DispatcherServlet is invoked again to handle the async‑generated return value.

By default, Callable uses SimpleAsyncTaskExecutor, which does not reuse threads. In practice a real AsyncTaskExecutor should be configured.

Implementation based on WebAsyncTask

WebAsyncTask

wraps a Callable and adds callbacks for timeout, error, and completion.

@GetMapping("/webAsyncTask")
public WebAsyncTask<String> webAsyncTask() {
    WebAsyncTask<String> result = new WebAsyncTask<>(30003, () -> "success");
    result.onTimeout(() -> {
        log.info("timeout callback");
        return "timeout callback";
    });
    result.onCompletion(() -> log.info("finish callback"));
    return result;
}

The timeout configured on WebAsyncTask overrides any global timeout setting.

Implementation based on DeferredResult

DeferredResult

works similarly to Callable but the actual result is set later from another thread.

private Map<String, DeferredResult<String>> deferredResultMap = new ConcurrentHashMap<>();

@GetMapping("/testDeferredResult")
public DeferredResult<String> testDeferredResult() {
    DeferredResult<String> deferredResult = new DeferredResult<>();
    deferredResultMap.put("test", deferredResult);
    return deferredResult;
}

@GetMapping("/testSetDeferredResult")
public String testSetDeferredResult() throws InterruptedException {
    DeferredResult<String> deferredResult = deferredResultMap.get("test");
    boolean flag = deferredResult.setResult("testSetDeferredResult");
    if (!flag) {
        log.info("Result already processed, operation ignored");
    }
    return "ok";
}

The client sees the request in a pending state until another thread calls setResult. The map can be managed by a dedicated object manager. It is advisable to clean up expired DeferredResult objects (using DeferredResult.isSetOrExpired()) to avoid memory leaks.

A timeout can also be set on a DeferredResult, which has higher priority than the global timeout.

Providing a Thread Pool for Async Requests

Async requests free the main Tomcat worker thread, allowing higher throughput under the same maximum request configuration. A custom thread pool should be supplied:

@Bean("mvcAsyncTaskExecutor")
public AsyncTaskExecutor asyncTaskExecutor() {
    ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
    executor.setCorePoolSize(5);
    executor.setMaxPoolSize(10);
    executor.setQueueCapacity(10);
    executor.setThreadNamePrefix("fyk-mvcAsyncTask-Thread-");
    executor.setRejectedExecutionHandler(new ThreadPoolExecutor.DiscardPolicy());
    executor.setWaitForTasksToCompleteOnShutdown(true);
    executor.setAwaitTerminationSeconds(30);
    executor.initialize();
    return executor;
}

Configure the executor in MVC async support:

@Configuration
public class FykWebMvcConfigurer implements WebMvcConfigurer {

    @Autowired
    @Qualifier("mvcAsyncTaskExecutor")
    private AsyncTaskExecutor asyncTaskExecutor;

    @Override
    public void configureAsyncSupport(AsyncSupportConfigurer configurer) {
        configurer.setDefaultTimeout(60001);
        configurer.setTaskExecutor(asyncTaskExecutor);
    }
}

When to Use Async Requests

Async requests improve throughput only when the maximum connections and worker threads are unchanged. They are not suitable for CPU‑bound endpoints that keep the thread busy for the entire request; in such cases increasing the worker thread pool is sufficient.

Ideal scenarios involve I/O‑bound work where the CPU is idle while waiting for external services (e.g., remote API calls). Moving the waiting to another thread frees the Tomcat worker thread to handle other requests.

Because async adds thread‑switching overhead, it may slightly increase individual request latency, but the impact is minimal compared to the throughput gain.