RocketMQ NameServer Architecture Design and Source Code Analysis

This article provides an in-depth analysis of RocketMQ's NameServer from a source code perspective, covering its architecture design, startup process, and route management mechanisms.

1. RocketMQ Architecture Overview

RocketMQ consists of four core components: NameServer, Broker, Producer, and Consumer. NameServer acts as a lightweight registration center, replacing ZooKeeper to provide routing information. Producers and consumers can query Topic-specific Broker IP lists through NameServer. Multiple NameServer instances form a cluster but operate independently without information exchange.

2. NameServer Architecture Design

NameServer manages routing information through three main mechanisms:

Route Registration : Broker servers send heartbeat signals to all NameServer instances during startup, and continue sending heartbeats every 30 seconds to maintain registration.

Route Removal : NameServer scans BrokerLiveTable every 10 seconds, removing brokers that haven't sent heartbeats for more than 120 seconds.

Route Discovery : NameServer does not actively push updates to clients. Instead, producers periodically pull the latest routing information, reducing complexity while maintaining high availability through client-side fault tolerance.

3. Startup Process

The NameServer startup involves two main steps: creating NamesrvController and then initializing and starting it. The initialization includes loading KV configurations, initializing the Netty server, setting up thread pools for network interaction, registering heartbeat mechanism threads (starting 5 seconds after launch, scanning every 10 seconds), and registering KV configuration printing threads.

4. Route Management - Routing Metadata

NameServer maintains five key data structures:

topicQueueTable : HashMap storing Topic message queue routing information for load balancing during message sending.

brokerAddrTable : HashMap storing Broker basic information including Broker Name, cluster name, and master/slave Broker addresses.

clusterAddrTable : HashMap storing Broker cluster information with all Broker names in the cluster.

brokerLiveTable : HashMap storing Broker status information, updated each time NameServer receives a heartbeat.

filterServerTable : HashMap storing FilterServer lists on Brokers for message filtering.

5. Route Registration Process

When a Broker registers, it performs seven key steps: acquiring a write lock to prevent concurrent writes to the routing table, checking if the Broker's cluster exists and creating it if needed, maintaining BrokerData, creating or updating Topic routing metadata if the Broker is Master with changed Topic configuration, updating BrokerLiveInfo, registering Broker's filterServer address list, and returning Master node information if the Broker is a slave.

6. Route Removal Process

Route removal is triggered by two conditions: NameServer scanning every 10 seconds and removing brokers with no heartbeat for 120 seconds, or when a Broker shuts down normally. The removal process involves six steps including removing from brokerLiveTable, filterServerTable, brokerAddrTable, clusterAddrTable, and topicQueueTable.

7. Route Discovery Process

Producers actively pull routing information from NameServer at regular intervals. NameServer returns a TopicRouteData object containing queueDatas, brokerDatas, filterServerTable, and optionally orderTopicConf for ordered messages.

Key Design Insights

The article highlights two notable design patterns: using JVM shutdown hooks for graceful shutdown, and employing read-write locks to allow concurrent reads while ensuring serial processing of broker heartbeats - a classic use case for read-write locks in high-concurrency scenarios.