Master RocketMQ 4.9.x Consumption: Architecture, Load Balancing, and Retry Strategies

1. Architecture Overview

RocketMQ 4.9.x consists of four core roles: NameServer – a lightweight, stateless routing registry; BrokerServer – stores, delivers and queries messages; Producer – publishes messages using load‑balancing to select a broker queue; and Consumer – pulls messages in either push or pull mode.

The cluster startup sequence is: start NameServer, start Broker (register with NameServer), create Topic, Producer obtains broker list from NameServer and sends messages, Consumer connects to NameServer, discovers broker locations, and begins pulling messages.

2. Publish‑Subscribe Model

RocketMQ uses a publish‑subscribe model with two consumption modes: Clustering (each message is consumed by only one consumer in a group) and Broadcasting (every consumer receives every message).

3. Storage Model

RocketMQ stores messages in a hybrid structure. All queues of a broker share a single commitlog file where messages are appended sequentially, enabling high write throughput and fast offset‑based lookup. A background thread builds consumequeue (consumer index) and indexfile files.

Consumequeue files are organized per topic‑queue, each entry holds the physical offset, size and tag hash, allowing consumers to locate messages without scanning the whole commitlog. Consumer progress is stored in consumerOffset.json as topic@group → logical offset mappings.

4. Consumption Process (Cluster Mode)

Cluster consumption is the most common mode. A consumer configures a consumer group , subscription topic , and a message listener , then calls start() . The workflow is:

Load‑balance assigns broker queues to each consumer instance.

Pull requests ( pullRequest ) are created and placed in a single‑threaded PullMessageService queue.

The pull service asynchronously fetches messages from the broker.

Fetched messages are stored in a processQueue (a red‑black tree) and handed to the consumer thread pool.

Consumer threads invoke the listener, handle success or failure, and update local offsets.

5. Load Balancing

Load balancing distributes broker queues among consumers in the same group. It runs when the consumer starts, every 20 seconds, and upon broker notifications. The algorithm sorts queues and consumer IDs, then applies the average‑allocation strategy (similar to pagination) to compute the queue set for each consumer.

6. Long Polling

Consumers pull messages, but to avoid excessive polling the broker holds pull requests in a pullRequestTable . Every 5 seconds a pullRequestHoldService checks whether new messages have arrived (by comparing the request’s offset with the queue’s max offset). If a new message is available, the request is re‑triggered, effectively implementing long polling and reducing latency.

7. Message Consumption Types

7.1 Concurrent Consumption

Messages are processed by a thread pool; order is not guaranteed. The flow:

Network callback stores messages in processQueue and builds a msgList .

Each msgList is wrapped into a ConsumeRequest and submitted to the thread pool.

Consumer threads invoke the listener, return results, and handle retries.

7.2 Ordered Consumption

For FIFO guarantees, each queue is consumed by a single thread. The consumer obtains a lock on the queue, pulls messages from the processQueue , and processes them sequentially. If a lock cannot be obtained, the consumer retries after 20 seconds. Failure handling includes retrying within the same queue or sending the message to a dead‑letter queue.

8. Progress Persistence

In cluster mode, consumers periodically commit offsets to the broker via the pull request and a dedicated consumer‑manager handler; the broker writes them to consumerOffset.json . In broadcast mode, each consumer persists offsets locally to offsets.json using LocalFileOffsetStore .

9. Retry Mechanism (Delay Messages)

When consumption fails, the client sends a CONSUMER_SEND_MSG_BACK request. The broker places the message into a retry topic named %RETRY%<group> with an increasing delay level (up to 16 attempts). After the maximum retries, the message is moved to a dead‑letter topic %DLQ%<group> . Delay levels are configured via messageDelayLevel and correspond to internal queues under SCHEDULE_TOPIC_XXXX .

<code>Message msg = new Message();
msg.setTopic("TopicA");
msg.setTags("Tag");
msg.setBody("this is a delay message".getBytes());
// delay level 5 → 1 minute
msg.setDelayTimeLevel(5);
producer.send(msg);
</code>

10. Summary

RocketMQ’s consumption logic features heavy client responsibilities (load balancing, pull, offset management, retry) and requires idempotent business handling to avoid duplicate consumption during scaling or broker failures. Understanding the architecture, storage model, and the detailed flow of pull‑based consumption helps developers build robust, high‑throughput messaging solutions.