How RocketMQ 5.0 Solves IoT Messaging Challenges with MQTT

Background and IoT Messaging Requirements

IoT device count is projected to exceed 200 billion by 2025, generating data with a growth rate of ~28 % and accounting for more than 90 % of real‑time streams. Edge computing will handle ~75 % of this data, so a messaging layer must support massive numbers of low‑capacity clients, unstable networks, and high fan‑out subscription patterns.

Differences Between Classic and IoT Messaging

Classic scenario : Brokers and clients run in data‑center or cloud servers with high bandwidth. Hundreds to thousands of servers produce hundreds to thousands of TPS each; consumer groups are modest (≤10 subscriptions per topic).

IoT scenario : Clients are tiny devices with limited CPU/memory, often behind weak or intermittent public networks. Device count can reach billions; each device generates low TPS, but a single message may need to be delivered to millions of subscribers, requiring massive subscription handling and low‑latency push.

RocketMQ‑MQTT Sub‑Product (RocketMQ 5.0)

RocketMQ‑MQTT adds three core capabilities for IoT:

Native support for the lightweight MQTT protocol with QoS 0/1/2 and flexible subscription modes, fully compatible with RocketMQ’s message model.

Storage‑compute separation: the traditional RocketMQ broker persists messages, while an MQTT Proxy layer handles massive connections, subscription management, and real‑time push, allowing independent scaling of proxy nodes.

Edge‑cloud integrated storage: each message is written once to the commit log and can be consumed by both edge devices and cloud applications, enabling unified stream storage and analytics.

Key Technical Designs

(1) IoT Message Storage Model

RocketMQ adopts a “read‑amplify‑first, write‑amplify‑second” model. All messages are written to a single public queue (read‑amplify), allowing millions of devices to read from the same queue and dramatically reducing storage cost. For wildcard subscriptions, an additional queue is created so that pattern‑based consumers can read from a dedicated queue. This hybrid model balances storage efficiency with the need for precise and wildcard matching.

(2) Edge‑Cloud Integrated Storage

All inbound messages—whether from classic RMQ/AMQP producers or MQTT devices—are written once to the commit log. Multiple logical queues (e.g., level‑1 service topics, multi‑level device topics) reference the same physical storage, providing a unified backend for both cloud services and edge IoT workloads.

(3) Massive Queue Scale Design

To support millions of queues, RocketMQ replaces per‑topic files with a single commit log and a compact CQ (Consume Queue) index stored in RocksDB. RocksDB’s sequential‑write performance and atomic WriteBatch enable high‑throughput index updates. Custom compaction logic uses the maximum physical offset to clean stale data, reducing random‑write overhead and allowing queue counts to scale to the million‑level.

(4) IoT Message Push Model

RocketMQ replaces classic long‑polling consumers with a hybrid push‑pull model. Devices connect via MQTT to an MQTT Proxy. When a new message arrives, a notify module generates an event that is sent to the Proxy. The Proxy matches the event against stored subscription data—either by broadcasting the event to all proxies or by consulting a lookup service for precise matching. Matched devices trigger a pull request to the broker, retrieve the message, and the Proxy pushes it downstream. A local cache in the Proxy reduces redundant reads during broadcast scenarios.