How TDMQ RocketMQ Implements Distributed Rate Limiting for High‑Throughput Messaging

Overview

With the rise of distributed system architectures, message queues have become core components for large‑scale, high‑concurrency online services. TDMQ RocketMQ version provides a high‑performance, highly reliable messaging middleware that uses a distributed rate‑limiting mechanism to dynamically adjust client send and consume speeds, ensuring stable operation under heavy load.

Conversion Rules

Normal message: sending or consuming one message counts as 1 TPS.

Advanced‑feature messages (delay, transaction, ordered, etc.): one message counts as 5 TPS.

Message size is measured in 4 KB units; each 4 KB (or partial) counts as 1 TPS.

Rate‑Limiting Behavior

TDMQ RocketMQ adopts a fast‑fail strategy: when a client’s request rate reaches the configured limit, the server immediately returns an error, allowing the client to detect throttling quickly. In a 1000 TPS cluster with a 1:1 send/consume ratio, the limits are 500 TPS for sending and 500 TPS for consuming. SDK log keywords are “Rate of message sending reaches limit…” and “Rate of message receiving reaches limit…”.

SDK retry policies differ:

5.x SDK: exponential back‑off retry, configurable max attempts (default 2). After exceeding max attempts, an exception is thrown.

4.x SDK: no automatic retry; the exception is thrown directly.

Pull‑message threads perform automatic back‑off retries.

Implementation Details

Two modes are supported:

Single‑node limiting : each node protects its own resources (CPU, memory, threads) from overload.

Distributed limiting : Proxy nodes request a token from a centralized Limiter service before processing SendMessage or PullMessage. If the token request fails, the request is rejected. The Limiter SDK embedded in the Proxy handles token acquisition and periodic usage reporting.

The default token counting period is 1 second; the article recommends extending it to 10 seconds to reduce throttling caused by short traffic spikes while keeping resource usage safe.

Client Practices

Estimate peak TPS based on current scale and future growth; reserve about 30 % headroom for bursts.

Isolate high‑stability workloads by using separate RocketMQ clusters (e.g., core transaction vs. log streams, production vs. testing).

Use the TDMQ console to monitor send/consume TPS; trigger alerts when either exceeds 70 % of capacity or when throttling occurs.

Enable Elastic TPS for bursty traffic. Example: a 4000 TPS professional cluster can automatically scale up to 6500 TPS, with clear cost rules for the elastic range.

Code Samples

4.x SDK – manual retry with exponential back‑off :

// Note: this is illustrative code only
final int maxAttempts = 3;
final int retryIntervalMillis = 200;
int attempt = 0;
do {
    try {
        SendResult sendResult = producer.send(message);
        log.info("Send message successfully, {}", sendResult);
        break;
    } catch (Throwable t) {
        attempt++;
        if (attempt >= maxAttempts) {
            log.warn("Failed to send message finally, run out of attempts, {}", t);
            break;
        }
        int waitMillis;
        if (t instanceof MQBrokerException && ((MQBrokerException) t).getResponseCode() == 215) {
            // Flow control error, exponential back‑off
            waitMillis = (int) Math.pow(2, attempt - 1) * retryIntervalMillis;
        } else {
            waitMillis = retryIntervalMillis;
        }
        log.warn("Retry after {} ms, attempt {}", waitMillis, attempt);
        try { Thread.sleep(waitMillis); } catch (InterruptedException ignore) {}
    }
} while (true);

5.x SDK – let SDK handle retries :

// Note: this is illustrative code only
Producer producer = provider.newProducerBuilder()
    .setClientConfiguration(clientConfiguration)
    .setTopics(topicName)
    .setMaxAttempts(3) // custom max attempts
    .build();
try {
    SendReceipt receipt = producer.send(message);
    log.info("Send message successfully, messageId={}", receipt.getMessageId());
} catch (Throwable t) {
    log.warn("Failed to send message", t);
    // Record failure for manual recovery
}

FAQ

Will messages be lost when throttled? Sending side discards the message; the client must handle the exception. Consuming side experiences delay but already stored messages are not lost.

Why does TPS appear larger than the raw message count? Advanced messages and larger payloads are converted to multiple TPS units; TPS is a peak‑value metric, while message count is an average.

Is occasional consumer throttling harmful? Generally no; it usually recovers quickly after client or server restarts or after scaling operations.

How to detect throttling? Look for SDK exceptions or log keywords, and monitor the “throttled produce TPS” and “throttled consume TPS” metrics on the TDMQ console.

Conclusion

TDMQ RocketMQ’s rate‑limiting mechanism safeguards high‑throughput online services by combining distributed token management with a fast‑fail strategy, balancing performance and precision. The system gracefully degrades to single‑node limiting if the Limiter service is unavailable, and developers can fine‑tune counting periods, enable elastic TPS, and monitor alerts to maintain stable operations.