Understanding RocketMQ Architecture: Components, Protocols, and Storage

RocketMQ System Architecture Overview

RocketMQ consists of four main components: NameServer (lightweight registry), Broker (message storage), Producer (supports sync, async, one‑way sending), and Consumer (supports push/pull modes).

Network Protocol

Before RocketMQ 5.0, client and broker/name server use the proprietary Remoting protocol (binary over Netty).

Since RocketMQ 5.0, the official SDK also supports gRPC while retaining the Remoting implementation.

Interaction Process

1. Broker registers and sends heartbeat to NameServer via Remoting (port 9876).

2. Client starts, connects to NameServer to obtain broker routing table.

3. Client communicates with Broker via Remoting (port 10911) or via Proxy (port 8081) using gRPC, which forwards to Broker.

Remoting is TCP‑based, while gRPC uses HTTP/2 over TCP.

Network Module

RocketMQ uses Netty as the underlying communication library, extending it with a Reactor model.

Broker has a Reactor boss thread listening for TCP connections and a worker thread pool (default 3 threads) handling network data, plus a SendMessage thread pool for disk writes (size adapts to CPU cores).

Storage Module

Data storage includes metadata and message data.

Metadata Storage

Broker metadata (name, ID, cluster, address, topic info) is stored in memory of each NameServer; brokers periodically heartbeat every 30 seconds, and NameServer detects failures if no heartbeat for 120 seconds.

Message Data Storage

Three file types are used: CommitLog (stores message bodies, default 1 GB per file), ConsumeQueue (index file with fixed‑length 20‑byte entries, ~5.7 MB per file), and IndexFile (key/time index, ~400 MB per file, holds ~20 M entries).

Flush Mechanism

Synchronous flush: ACK returned after the message is persisted to disk.

Asynchronous flush: ACK returned after write to page cache; a background thread flushes to disk.

Producer and Consumer Mechanics

Producer

Producer connects to a NameServer, obtains broker routing, selects a MessageQueue via load balancing, and sends messages. It supports one‑way, synchronous, and asynchronous sending.

Consumer

Consumer also connects to a NameServer, gets broker info, and can consume via Pull, Push, or Pop (introduced in 5.0). Consumers belong to a consumer group; in cluster mode, any consumer in the group can process a message, while in broadcast mode all consumers receive each message.

Transactional Messages

RocketMQ provides transactional messages using a two‑phase commit (2PC) plus compensation. Steps: (1) Producer sends a half‑transaction message; (2) Broker persists it and ACKs; (3) Producer executes local transaction; (4) Producer commits or rolls back; (5‑7) If the final decision is lost, the broker performs a message check and the producer re‑submits the decision.

Summary

RocketMQ supports both Remoting and gRPC protocols.

The network layer is built on Netty.

The storage layer uses sequential file storage with CommitLog, ConsumeQueue, and IndexFile.

Brokers register metadata to NameServers.

Producers obtain broker info from NameServer and send to selected MessageQueue.

Consumers retrieve broker info similarly and consume messages, tracking offsets.

Consumer groups enable load‑balanced or broadcast consumption.