Understanding Apache RocketMQ: Architecture, Components, and Key Concepts

Overall Architecture

RocketMQ is an open‑source high‑performance, high‑throughput distributed messaging middleware from Alibaba. It consists of Producer, Broker, and Consumer. Producers generate messages, Consumers retrieve them, and Brokers store and forward messages. Each Broker can hold multiple Topics, which may be sharded across a Broker group.

Namesrv

Namesrv provides service registration and discovery for producers and consumers, offering routing information for topics. It manages broker nodes, monitors their status, and handles route registration and removal (fault exclusion). Namesrv is a stateless node; multiple instances form an independent cluster without exchanging information.

Routing metadata

topicQueueTable: topic queue routing info

brokerAddrTable: basic broker info (name, cluster, master address)

clusterAddrTable: broker cluster information

brokerLiveTable: broker status

filterServerTable: list of filter servers for message filtering

Routing registration occurs via broker heartbeats sent to all Namesrv instances every 30 seconds. Routing deletion is triggered when a broker’s heartbeat is absent for 120 seconds, causing the broker’s routing data to be removed.

Routing discovery is non‑real‑time; clients periodically pull the latest routing data from Namesrv.

Broker

Broker acts as a message relay, storing and forwarding messages. It maintains metadata such as consumer groups, offsets, topics, and queues. Brokers are organized in groups with master and slave nodes; slaves replicate data from masters. Each broker establishes long‑lived connections to all Namesrv nodes and sends heartbeats.

Producer

Producer is stateless and can be clustered. It connects to a random Namesrv node to obtain topic routing, then establishes a long‑lived connection to the master broker of the target topic. RocketMQ supports synchronous, asynchronous, ordered, and one‑way sending modes.

Consumer

Consumer also connects to a random Namesrv node to fetch routing information and then to the appropriate master and slave brokers. It can pull messages or receive them via a push‑style long‑polling mechanism. In cluster mode, each queue is assigned to a single consumer instance within a consumer group.

Key Concepts

Topic: a collection of messages, possibly split into multiple queues.

Producer Group: a set of producers with identical logic.

Consumer Group: a set of consumers that share load and provide fault tolerance.

Normal Ordered Message vs. Strictly Ordered Message.

Message: the smallest unit, identified by a unique Message ID and optional Key.

Tag: a label used to differentiate message types within the same topic.

Message Middleware Fundamentals

Message middleware solves asynchronous processing and traffic shaping. Core capabilities include publish, subscribe, and consume. High availability and reliability are achieved through persistent storage (single CommitLog per broker) and master/slave replication. Distributed scalability is provided by topic partitioning and dynamic expansion.

Message Priority

RocketMQ does not natively support priority; a workaround is to create separate high‑priority and normal‑priority queues.

Message Ordering

Strict ordering is guaranteed within a queue. Producers can select a specific queue using MessageQueueSelector, and consumers can allocate queues using AllocateMessageQueueStrategy.

Parallelism

Production parallelism depends on broker count; consumption parallelism depends on the number of queues and consumer threads. Broadcast mode delivers all messages to every consumer, while clustering mode distributes queues among consumers.

Queue Allocation Strategies

Built‑in selectors include SelectMessageQueueByMachineRoom, SelectMessageQueueByHash, and SelectMessageQueueByRandom. Built‑in allocation strategies include AllocateMessageQueueAveragely, AllocateMessageQueueAveragelyByCircle, AllocateMachineRoomNearby, AllocateMessageQueueByMachineRoom, AllocateMessageQueueConsistentHash, and AllocateMessageQueueByConfig.