How to Deploy a High‑Availability RocketMQ Cluster on Kubernetes with Helm

1. Deployment Architecture Design

In Kubernetes, a RocketMQ cluster is typically deployed as a StatefulSet to provide stable network identities and persistent storage. A recommended production topology includes multiple NameServer replicas, a set of Broker replicas (2 masters and 2 slaves), and a Dashboard service.

Topology: Producer/Consumer → NameServer Service → Broker StatefulSet → PVC storage.

2. Deployment Method Choice

Two main approaches are supported:

Helm Chart (recommended)

Quick installation with rich configurability.

Official or community‑maintained charts are available.

Native YAML

Suitable for deep customisation.

Higher maintenance cost.

3. Helm Deployment

3.1 Add Helm repository

helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update
# Bitnami does not provide an official RocketMQ chart; use streamnative/rocketmq or a community chart:
helm repo add streamnative https://charts.streamnative.io
helm repo update

3.2 Create values.yaml

Example configuration (2 NameServer, 2 Master, 2 Slave, HostPath persistence):

nameserver:
  replicaCount: 2
  persistence:
    enabled: true
    storageClass: ""
    accessMode: ReadWriteOnce
    size: 5Gi
    hostPath: /data/rocketmq/nameserver
broker:
  replicaCount: 4
  config:
    brokerClusterName: rocketmq-cluster
    deleteWhen: "04"
    fileReservedTime: "48"
    flushDiskType: ASYNC_FLUSH
  persistence:
    enabled: true
    storageClass: ""
    accessMode: ReadWriteOnce
    size: 20Gi
    hostPath: /data/rocketmq/broker
  service:
    type: NodePort
    ports:
      broker: 30911
      admin: 30909
dashboard:
  enabled: true
  service:
    type: NodePort
    nodePort: 30880

3.3 Install

helm install rocketmq streamnative/rocketmq -f values.yaml

3.4 Verify

kubectl get pods -l app.kubernetes.io/name=rocketmq
kubectl logs rocketmq-nameserver-0 -c nameserver
kubectl logs rocketmq-broker-0 -c broker

4. Native YAML Deployment (Simplified)

4.1 NameServer Deployment + Service

... (same values as above for nameserver) ...

4.2 Broker StatefulSet + Service

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: rmq-broker
spec:
  serviceName: rmq-broker
  replicas: 4
  selector:
    matchLabels:
      app: rmq-broker
  template:
    metadata:
      labels:
        app: rmq-broker
    spec:
      containers:
      - name: broker
        image: apache/rocketmq:5.2.0
        command: ["sh","mqbroker","-n","rmq-nameserver:9876","-c","/home/rocketmq/conf/broker.conf"]
        ports:
        - containerPort: 10911
        volumeMounts:
        - name: data
          mountPath: /home/rocketmq/store
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: ["ReadWriteOnce"]
      resources:
        requests:
          storage: 20Gi

5. External Access Configuration

NodePort – simple direct host‑port mapping.

LoadBalancer – recommended for cloud environments.

Ingress TCP – Nginx/Traefik TCP forwarding support.

6. Persistence Recommendations

Production should use a StorageClass (e.g., Ceph, NFS, Longhorn).

For testing, a hostPath mount is sufficient.

7. Monitoring & Management

Deploy RocketMQ Dashboard (exposed via NodePort).

Integrate Prometheus + Grafana to monitor message backlog, TPS, latency, etc.

8. Production‑grade Helm Values (Annotated)

# values-prod.yaml
global:
  storageClass: "ebs-ssd"   # AWS EBS SSD / Alibaba Cloud ESSD
nameserver:
  replicaCount: 3
  podAntiAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
    - labelSelector:
        matchExpressions:
        - key: app.kubernetes.io/component
          operator: In
          values: ["nameserver"]
      topologyKey: "topology.kubernetes.io/zone"
  resources:
    requests:
      cpu: "1"
      memory: "2Gi"
broker:
  replicaCount: 4
  config:
    brokerClusterName: "rocketmq-prod-cluster"
    brokerName: "default-broker"
    brokerId: "0"   # Master = 0, Slave = 1
    brokerRole: "ASYNC_MASTER"
    flushDiskType: "ASYNC_FLUSH"
    diskMaxUsedSpaceRatio: "75"
    transactionTimeOut: "3000"
  podManagementPolicy: "Parallel"
  podAntiAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
    - weight: 100
      podAffinityTerm:
        labelSelector:
          matchExpressions:
          - key: app.kubernetes.io/component
            operator: In
            values: ["broker"]
        topologyKey: "kubernetes.io/hostname"
  persistence:
    size: "100Gi"
    accessModes: ["ReadWriteOnce"]
dashboard:
  enabled: true
  ingress:
    enabled: true
    hosts:
    - rocketmq-dashboard.example.com

9. One‑click Deployment Command (with Health Checks)

# Install Helm chart with dependency checks
helm upgrade --install rocketmq streamnative/rocketmq \
  -f values-prod.yaml \
  --namespace rocketmq \
  --create-namespace \
  --wait \
  --timeout 10m

# Verify deployment status
kubectl -n rocketmq get pods -l app.kubernetes.io/name=rocketmq -o wide

# Test message send/receive (requires mqadmin tool)
kubectl exec -n rocketmq rocketmq-broker-0 -- sh tools.sh org.apache.rocketmq.example.quickstart.Producer

10. Monitoring Alert Plan (Prometheus + Grafana)

Expose metrics port – enable broker metrics and ServiceMonitor.

Key metrics and thresholds :

rocketmq_broker_tps < 100 for 5 min – throughput drop.

rocketmq_consumer_lag > 5000 – severe message backlog.

rocketmq_disk_commitlog_ratio > 80% – disk space low.

rocketmq_pending_nonpersistent > 1000 – non‑persistent message buildup.

Import official Grafana dashboard:

kubectl apply -f https://raw.githubusercontent.com/apache/rocketmq-exporter/master/grafana/RocketMQ-Overview.json

11. Security Hardening

TLS encryption – enable TLS and provide a secret containing certificates.

ACL access control – create a secret with ACL configuration.

Network policies – restrict access to specific namespaces.

12. Troubleshooting Toolbox

# View broker runtime configuration
kubectl exec -n rocketmq rocketmq-broker-0 -- sh mqadmin getBrokerConfig

# Check message backlog
kubectl exec -n rocketmq rocketmq-broker-0 -- sh mqadmin consumerProgress

# Force clean expired files (use with caution)
kubectl exec -n rocketmq rocketmq-broker-0 -- sh mqadmin cleanExpiredCQ