Mastering Rate Limiting in Java with Semaphore: A Practical Guide

Rate limiting is a technique that restricts the number of requests to prevent system overload and ensure stability.

Before a promotional event, traffic is estimated, stress‑tested, and if performance is insufficient, optimization or scaling is performed. During the event, if actual traffic exceeds estimates, degradation or rate limiting is applied to sacrifice some users for overall stability.

The following simple approach, inspired by a Taobao engineer, uses a semaphore to enforce a maximum concurrent request count.

First, define the upper limit. When a request arrives, check whether the current count exceeds the limit; if so, reject the request, otherwise process it and increment the count. After processing, decrement the count.

Java provides Semaphore for this purpose.

Example code:

// Define semaphore with 100 permits
Semaphore semaphore = new Semaphore(100);

// If there are waiting threads, reject new request
if (semaphore.getQueueLength() > 0) {
    return;
}

try {
    // Acquire a permit
    semaphore.acquire();

    // ... process business logic ...

} catch (Exception e) {
    // handle exception
} finally {
    // Release the permit
    semaphore.release();
}

With 100 permits, the 101st concurrent request will wait in the queue and not be processed until a permit is released.