Sync vs Async: Achieving 10× Throughput in SpringBoot APIs

Introduction

Before Servlet 3.0 each HTTP request was processed by a dedicated thread from start to finish. Servlet 3.0 introduced asynchronous processing, allowing the container‑allocated thread to be released while the request remains open, which can increase overall throughput.

Implementation Options in SpringBoot

SpringBoot provides four ways to create asynchronous endpoints: AsyncContext, Callable, WebAsyncTask, and DeferredResult. The article focuses on the latter three.

Using Callable<String>

Returning a Callable from a controller signals Spring MVC to start asynchronous processing. The flow is:

Controller returns a Callable.

Spring MVC invokes request.startAsync() and submits the Callable to an AsyncTaskExecutor.

The original servlet thread and filters exit, keeping the response open.

When the Callable produces a result, Spring MVC dispatches the request back to the servlet container to complete the response.

By default Spring uses SimpleAsyncTaskExecutor, which creates a new thread for each task; in production a custom AsyncTaskExecutor should be supplied.

Using WebAsyncTask<String>

WebAsyncTask

wraps a Callable and adds callbacks for timeout, error, and completion. Example code demonstrates setting a 30‑second timeout, a timeout callback, and a completion callback before returning the task.

@GetMapping("/webAsyncTask")
public WebAsyncTask<String> webAsyncTask() {
    WebAsyncTask<String> result = new WebAsyncTask<>(30003, () -> "success");
    result.onTimeout(() -> {
        log.info("timeout callback");
        return "timeout callback";
    });
    result.onCompletion(() -> log.info("finish callback"));
    return result;
}

The timeout configured on the task overrides any global timeout setting.

Using DeferredResult<String>

DeferredResult

separates request handling from result production. The controller creates a DeferredResult and stores it in a shared map. Another endpoint later retrieves the object and calls setResult, which completes the pending request.

private Map<String, DeferredResult<String>> deferredResultMap = new ConcurrentHashMap<>();

@GetMapping("/testDeferredResult")
public DeferredResult<String> testDeferredResult() {
    DeferredResult<String> deferredResult = new DeferredResult<>();
    deferredResultMap.put("test", deferredResult);
    return deferredResult;
}

@GetMapping("/testSetDeferredResult")
public String testSetDeferredResult() throws InterruptedException {
    DeferredResult<String> deferredResult = deferredResultMap.get("test");
    boolean flag = deferredResult.setResult("testSetDeferredResult");
    if (!flag) {
        log.info("Result already processed, operation ignored");
    }
    return "ok";
}

When using DeferredResult, developers should periodically clean up expired or already‑set objects (e.g., via DeferredResult.isSetOrExpired()) to avoid memory leaks.

Configuring a Dedicated Thread Pool

Because asynchronous requests free the servlet thread, a custom thread pool is recommended for executing the background tasks. The article provides a sample bean definition and shows how to plug it into Spring’s async support.

@Bean("mvcAsyncTaskExecutor")
public AsyncTaskExecutor asyncTaskExecutor() {
    ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
    executor.setCorePoolSize(5);
    executor.setMaxPoolSize(10);
    executor.setQueueCapacity(10);
    executor.setThreadNamePrefix("fyk-mvcAsyncTask-Thread-");
    executor.setRejectedExecutionHandler(new ThreadPoolExecutor.DiscardPolicy());
    executor.setWaitForTasksToCompleteOnShutdown(true);
    executor.setAwaitTerminationSeconds(30);
    executor.initialize();
    return executor;
}

@Configuration
public class FykWebMvcConfigurer implements WebMvcConfigurer {
    @Autowired
    @Qualifier("mvcAsyncTaskExecutor")
    private AsyncTaskExecutor asyncTaskExecutor;

    @Override
    public void configureAsyncSupport(AsyncSupportConfigurer configurer) {
        // 0 or negative means never timeout
        configurer.setDefaultTimeout(60001);
        configurer.setTaskExecutor(asyncTaskExecutor);
    }
}

When to Use Asynchronous Requests

Async processing improves throughput only when the request’s business logic spends most of its time waiting (e.g., calling external services). If the work is CPU‑bound, moving it to another thread does not increase capacity; simply increasing the servlet thread pool is more effective. Although async adds a small overhead due to thread switching, the latency increase is negligible compared with the throughput gains.

Note: From the client’s perspective the response is identical for synchronous and asynchronous endpoints; the only visible difference may be a slightly longer response time for async calls.