How to Throttle Java Virtual Threads with Semaphores for Optimal Throughput

Virtual threads are one of the most attractive features of Java 21, greatly simplifying and enhancing concurrency in Java applications. However, managing the resulting throughput is essential.

In most cases developers do not need to create virtual threads manually; web frameworks like Tomcat or Jetty automatically generate a virtual thread for each incoming request.

When custom concurrency is required, developers can use the method introduced in the Java 21 feature article, such as Executors.newVirtualThreadPerTaskExecutor(). Example:

Runnable runnable = () -> {
    System.out.println("Hello, www.didispace.com");
};

try (ExecutorService executorService = Executors.newVirtualThreadPerTaskExecutor()) {
    for (int i = 0; i < 100; i++) {
        executorService.submit(runnable);
    }
}

To control the throughput of virtual threads, the answer is a semaphore. Key point: do not pool virtual threads because they are not scarce resources. The best solution is to use java.util.concurrent.Semaphore.

Virtual Thread Throttling

The following code demonstrates how to use a semaphore to limit the number of concurrent virtual threads:

public class SemaphoreExample {

    // Define a semaphore limiting concurrency to 10
    private static final Semaphore POOL = new Semaphore(10);

    public void callOldService(...) {
        try {
            POOL.acquire(); // try to acquire a permit
        } catch (InterruptedException e) {
            // handle acquisition failure
        }

        try {
            // business logic executed by a virtual thread
        } finally {
            POOL.release(); // release the permit
        }
    }
}