What’s New in RocketMQ 5.x? A Deep Dive into Cloud‑Native Features, Proxy, Pop, and Tiered Storage

As cloud‑native applications become the backbone of digital transformation, a robust, scalable messaging system is essential. This article, based on a presentation by Tencent Cloud senior engineer Li Wei, examines the major innovations introduced in Apache RocketMQ 5.x and how they are applied on Tencent Cloud.

New Features in RocketMQ 5.x

RocketMQ 5.x enhances core functionalities that users rely on and adds several brand‑new capabilities. The enhancements improve performance, scalability, and ease of use, while the new features broaden application scenarios.

Architecture Changes

The 5.x architecture adds a Proxy layer between clients and the traditional NameServer/Broker stack. Deployment order is now Namesrv → Controller → Broker → Proxy → producers/consumers. Two client types exist:

TCP‑based producers using the classic Remoting protocol (as in 4.x).

gRPC clients (called RocketMQ Clients) that communicate via the Proxy.

In 5.x the client no longer holds routing information; instead, the Proxy fetches routes from NameSrv and forwards messages to the appropriate Broker.

Proxy Component

The Proxy handles client connections, route calculation, and message lease management. It introduces a three‑layer hierarchy: user layer (produce/consume), Proxy (compute) layer, and storage layer (NameSrv + Broker). Benefits include:

Separation of compute and storage, enabling on‑demand resource scaling.

Unified gRPC interface supporting multiple languages (C++, Python, Go, etc.).

Simplified network topology—clients only need to reach the Proxy, reducing configuration complexity.

Pop Consumption Model

Traditional push consumers can become stuck, causing message backlog. The Pop model introduces a gRPC Pop request to the Proxy, which then uses a Pop protocol to fetch messages from the Broker. Key advantages:

Clients are stateless and simple, relying on gRPC.

When one consumer instance stalls, other instances can still pull pending messages, improving reliability.

Built‑in retry logic based on random selection of unprocessed messages.

ACK Process

After a client processes a message, it sends an ACK. The system generates a checkpoint and a message handle (BrokerName‑QID‑offset). The ACK flow includes in‑memory matching and persistent checkpoint storage. If the client is slow, the checkpoint is persisted to a dedicated topic for asynchronous matching.

Delayed and Timer Messages

Version 4.x used a fixed Schedule_Topic_XXXX with up to 16 delay levels. RocketMQ 5.x replaces this with a timer‑wheel implementation that supports arbitrary delay dimensions and longer intervals (beyond 48 hours). Messages are written to a Timer Topic, transformed into timer‑wheel slots, and later triggered by a scheduler thread.

Additional 5.x Features

Dledger and Dledger Controller for replicated commit logs.

Request‑reply semantics for reliable delivery.

Broker Container to run multiple brokers in one process.

Logical queues, static/dynamic topics, lightweight queues, and KV‑based compaction topics.

MQTT support for millions of topics and devices.

OpenTelemetry integration for metrics, logs, and tracing.

Containerized Deployment on Tencent Cloud

RocketMQ is offered as a Serverless service on Tencent Cloud’s TKE Serverless platform. Containerization solves delivery challenges by standardizing process lifecycle, resource isolation (cgroups, namespaces), and cross‑AZ/Region deployment. Key considerations include:

Pay‑as‑you‑go resource billing.

Fixed IP support and VPC‑CNI networking.

Configuration consistency via ConfigMaps and Admin API.

Persistent storage using PVCs to retain data across restarts.

Network Complexity

RocketMQ clusters must serve diverse environments (different VPCs, public internet, on‑prem, other clouds). A unified routing list is provided to clients, and the system must keep routing information up‑to‑date as network policies change.

Tiered Storage Practice

Version 5.x introduces a tiered storage layer that offloads older or cold data to object storage. A new Dispatcher asynchronously writes data to the tiered store while preserving the original commit‑log flow. Consumers first check the in‑memory cache; if insufficient, they read from the tiered store via the Dispatcher.

Distributed Rate‑Limiting

To protect multi‑tenant clusters, RocketMQ adds a distributed rate‑limiting mechanism. A central manager distributes quotas to each Proxy, enabling global or local throttling based on tags, priority, and fallback strategies (fast‑fail or queue). This mitigates hotspot issues on heavily loaded Proxies.

Future Outlook

The author plans to continue community evangelism, publish more technical articles, and contribute code back to the open‑source project. The Serverless 5.x version is already live, and further enhancements—such as Pop‑based gateway functions, unified authentication, and cancellation of pending timer messages—are under consideration.

Overall, RocketMQ 5.x represents a significant step toward a fully cloud‑native, scalable, and observable messaging platform suitable for large‑scale production environments.