How to Set Up a High‑Availability RocketMQ Cluster on Two Servers (2‑Master‑2‑Slave Async)

RocketMQ Overview

RocketMQ is an open‑source messaging middleware released by Alibaba in 2012 and graduated to an Apache top‑level project in 2017. It is used by thousands of applications and can handle tens of millions of messages per second during peak events. The design is similar to Kafka but adds NameServer‑based service discovery, zero‑copy I/O, and flexible clustering modes.

Key Features

Cluster model with load balancing and horizontal scalability

Billions‑level message backlog capacity

Zero‑copy sequential writes and random reads

Netty NIO communication framework

NameServer replaces Zookeeper for service routing

Message retry, query, and high‑availability guarantees

High‑Availability Architecture

RocketMQ supports several deployment patterns:

Single Master – simple but a single point of failure.

Multiple Masters – highest performance, but may lose messages if a master restarts.

Multiple Masters with Asynchronous Replication (Master‑Slave) – near‑master performance with transparent failover.

Multiple Masters with Synchronous Replication – highest data safety with a slight performance cost.

This guide implements the fourth pattern (two masters, two slaves, synchronous replication).

Port Planning

Two cloud servers are used. The following ports are reserved for NameServer and brokers:

# Server 1 (42.192.77.73)
NameServer1   9876
BrokerA‑master 10910
BrokerB‑slave   10921

# Server 2 (39.103.144.86)
NameServer2   9876
BrokerB‑master 10920
BrokerA‑slave   10911

Downloading and Extracting RocketMQ

cd /jackxu
wget https://mirror.bit.edu.cn/apache/rocketmq/4.8.0/rocketmq-all-4.8.0-bin-release.zip
unzip rocketmq-all-4.8.0-bin-release.zip
mv rocketmq-all-4.8.0-bin-release rocketmq

Configuring Brokers

Three configuration profiles exist under rocketmq/conf: 2m-2s-async – two masters, two slaves, asynchronous replication (used in this guide) 2m-2s-sync – two masters, two slaves, synchronous replication 2m-noslave – two masters only

For each broker edit the corresponding *.properties file to set the cluster name, listening port, auto‑creation flags, NameServer addresses, and storage paths. Example for broker-a.properties on the first server:

brokerClusterName=jackxu-cluster
listenPort=10910
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
namesrvAddr=39.103.144.86:9876;42.192.77.73:9876
storePathRootDir=/jackxu/rocketmq/store/broker-a
storePathCommitLog=/jackxu/rocketmq/store/broker-a/commitlog
storePathConsumeQueue=/jackxu/rocketmq/store/broker-a/consumequeue
storePathIndex=/jackxu/rocketmq/store/broker-a/index
storeCheckpoint=/jackxu/rocketmq/store/checkpoint
abortFile=/jackxu/rocketmq/store/abort

Make analogous changes for broker-b-s.properties, broker-b.properties, and broker-a-s.properties on the second server, adjusting listenPort and storage directories accordingly.

Creating Data Directories

mkdir -p /jackxu/rocketmq/store/broker-a /jackxu/rocketmq/store/broker-a/consumequeue /jackxu/rocketmq/store/broker-a/commitlog /jackxu/rocketmq/store/broker-a/index /jackxu/rocketmq/logs
mkdir -p /jackxu/rocketmq/store/broker-b /jackxu/rocketmq/store/broker-b/consumequeue /jackxu/rocketmq/store/broker-b/commitlog /jackxu/rocketmq/store/broker-b/index /jackxu/rocketmq/logs

Starting NameServers

nohup sh /jackxu/rocketmq/bin/mqnamesrv > /jackxu/rocketmq/logs/mqnamesrv.log 2>&1 &
# Verify with:
 tail -f /jackxu/rocketmq/logs/mqnamesrv.log

Starting Brokers (order matters)

# Server 1 – Master A
nohup sh /jackxu/rocketmq/bin/mqbroker -c /jackxu/rocketmq/conf/2m-2s-sync/broker-a.properties > /jackxu/rocketmq/logs/broker-a.log 2>&1 &
# Server 2 – Slave A
nohup sh /jackxu/rocketmq/bin/mqbroker -c /jackxu/rocketmq/conf/2m-2s-sync/broker-a-s.properties > /jackxu/rocketmq/logs/broker-a-s.log 2>&1 &
# Server 2 – Master B
nohup sh /jackxu/rocketmq/bin/mqbroker -c /jackxu/rocketmq/conf/2m-2s-sync/broker-b.properties > /jackxu/rocketmq/logs/broker-b.log 2>&1 &
# Server 1 – Slave B
nohup sh /jackxu/rocketmq/bin/mqbroker -c /jackxu/rocketmq/conf/2m-2s-sync/broker-b-s.properties > /jackxu/rocketmq/logs/broker-b-s.log 2>&1 &
# Verify processes (should show four Java processes):
 jps

Troubleshooting Common Issues

Missing JDK – RocketMQ runs on Java; install a JDK before starting.

Insufficient memory – Reduce the memory settings in runserver.sh and runbroker.sh for small instances (e.g., 1 CPU, 2 GB).

Firewall / Port blocking – Disable firewalld or open port 9876 (NameServer) and the broker ports.

Incorrect internal IP – Add the public IP to brokerIP1 in the broker config to avoid accidental connections to internal cloud addresses.

Web Console Installation

The official RocketMQ console source is hosted at https://github.com/apache/rocketmq-externals. Download and build it:

cd /jackxu
wget https://github.com/apache/rocketmq-externals/archive/master.zip
unzip master.zip
cd rocketmq-externals-master/rocketmq-console/src/main/resources
vim application.properties
# Change port and NameServer list:
server.port=7298
rocketmq.config.namesrvAddr=39.103.144.86:9876;42.192.77.73:9876

Build the console with Maven and run the JAR:

cd /jackxu/rocketmq-externals-master/rocketmq-console
mvn clean package -Dmaven.test.skip=true
cd target
java -jar rocketmq-console-ng-2.0.0.jar

Access the UI at http://42.192.77.73:7298/. The dashboard provides sections for Operations, Cluster, Topics, Consumers, Producers, Messages, and User Center. The most frequently used views are Cluster, Topics, Consumers, and Messages.

Configuration File Details

All brokers share the same brokerClusterName=jackxu-cluster and the same NameServer address list. brokerId=0 denotes a master, brokerId=1 a slave. Important properties: brokerRole – ASYNC_MASTER or SYNC_MASTER for masters, SLAVE for slaves. flushDiskType – ASYNC_FLUSH (default, higher throughput) or SYNC_FLUSH (no data loss).

High‑Availability & Failover (DLedger)

Since version 4.5.0 RocketMQ supports DLedger (Raft‑based) for automatic leader election without external services. To enable DLedger, add the following to the broker configuration:

# Enable DLedger
enableDLegerCommitLog=true
dLegerGroup=broker-a
dLegerPeers=n0-192.168.44.163:10911;n1-192.168.44.164:10911;n2-192.168.44.165:10911
dLegerSelfId=n0

The recommended production configuration combines asynchronous flushing with synchronous replication.

Shutdown Procedure

Stop the brokers first, then the NameServers:

cd /jackxu/rocketmq/bin
sh mqshutdown broker
sh mqshutdown namesrv