Introduction to MQTT and JMQ’s MQTT Support Architecture

Message Queuing Telemetry Transport (MQTT) is a lightweight client/server publish‑subscribe messaging protocol originally proposed by IBM and now an OASIS standard, ideal for low‑power, bandwidth‑constrained scenarios such as machine‑to‑machine (M2M) and Internet of Things (IoT) communications.

MQTT Features

Publish/subscribe model provides one‑to‑many message distribution and decouples applications.

Payload content is opaque to the transport layer.

Three levels of Quality of Service (QoS) are defined.

Minimal transmission overhead reduces network traffic.

Connection failures trigger notifications to relevant parties.

Challenges Addressed by JMQ

Clustered deployment and automatic fault‑recovery while remaining compatible with native MQTT clients.

Efficient bulk subscription/unsubscription to save network bandwidth and support pattern‑based subscriptions.

Support for multiple QoS levels, with JMQ guaranteeing at least once delivery.

Push‑mode publishing to further reduce consumer latency.

Lightweight deployment and streamlined configuration for small, independent clusters.

System Architecture

JMQ supports MQTT 3.1.1 by adding an MQTT gateway on top of its core, exposing services to Internet clients. The architecture consists of a connection layer, session layer, and additional components as shown in the diagram.

MQTT Proxy Layered Design

1) Connection Layer – Handles TCP connections, MQTT encoding/decoding, heartbeat detection, message processing, concurrent connection limits, and flow control.

2) Session Layer – Manages publish/subscribe interactions, caches client subscriptions, terminates subscriptions, and processes QoS 0/1 messages with retransmission support.

MQTT Message Flow

JMQ Broker operates in pull mode; to achieve real‑time delivery, a Long‑Pull mechanism pushes messages from Broker to Proxy, maintaining long connections with consumers. For QoS 1, only messages within a flight window are pulled to avoid backlog.

Both QoS 0 and QoS 1 are supported: QoS 0 messages are automatically acknowledged on broker retrieval, and acknowledgments between Consumer‑Proxy and Proxy‑Broker are parallel.

Proxy Statelessness

Proxy maintains full subscription state.

Consumer‑Proxy connection health determines the lifecycle of the Broker‑Consumer‑Proxy relationship.

Clients can switch among multiple proxies.

Broker Automatic Election

Based on Raft consensus for leader election.

Message and Packet IDs

MQTT defines a 16‑bit packet ID for client‑server exchange; after entering the message server, the packet is converted to an internal message ID.

Subscription Process

Subscription relationships are stored both as ClientId‑TopicPattern and ClientId‑Topic mappings, enabling O(1) lookup of clients for a given topic. When a new topic matches an existing pattern, the corresponding client is automatically subscribed and the message is pushed to its active session.

Subscription changes are propagated via long‑lived connections between Proxy and EventNotification, with EventNotification employing a primary‑secondary architecture and Proxy periodically pulling updated subscription data from storage.

JMQ MQTT Advantages

Full compatibility with MQTT 3.1.1.

Robust clustering, containerized deployment, elastic scaling, and transparent horizontal expansion.

Comprehensive operations suite: one‑click onboarding, real‑time monitoring, customizable alerts.

Automated fault‑recovery mechanisms.

Persistent message storage ensuring high reliability.

Performance capable of handling millions of concurrent connections.

JMQ MQTT in JD.com

The MQTT service is already deployed in JD’s offline fresh‑food stores to support “shopping‑following‑cart” scenarios, with plans to extend to broader IoT domains such as autonomous vehicles, warehouses, smart speakers, and refrigerators, integrating with compute and analytics platforms for greater business value.