Understanding MQTT: History, Architecture, and Message Format Explained

I have been reading about the MQTT protocol and have compiled some information and my own understanding. Please correct any errors, and feel free to discuss. This article briefly outlines MQTT's history, the problems it addresses, and the basic protocol format.

Preface

MQTT is a simple hub‑and‑spoke system: communication between sensors, applications, and devices is achieved through a central broker service. Its lightweight and low‑bandwidth characteristics make it suitable for many applications, such as home automation (HVAC, lighting, smart devices, security, etc.).

1. History

The MQTT protocol was invented in 1999 by Andy Stanford‑Clark (IBM) and Arlen Nipper (Arcom, now Cirrus Link). They needed a protocol that minimized battery consumption and bandwidth for satellite communication with oil pipelines. The original requirements were:

Simple implementation

Quality‑of‑service data transmission

Lightweight and bandwidth‑efficient

Data obliviousness

Persistent session awareness

2. Protocol Format

MQTT control packets consist of three parts: a fixed header, an optional variable header, and an optional payload. The fixed header is present in all packets, while the variable header and payload appear only in certain packet types.

Fixed Header

The fixed header is two bytes long. Its format is shown below.

Control Packet Types

The first byte bits 7‑4 (an unsigned integer) indicate the control packet type. The mapping is illustrated below.

Flags

The first byte bits 3‑0 contain flags specific to each packet type. These flags must follow the specification; otherwise, the receiver must disconnect.

Remaining Length

The second byte (and subsequent bytes) encode the length of the remaining part of the packet (variable header + payload) using a variable‑length scheme up to four bytes. For values < 128 a single byte is used; larger values use continuation bits. Example: decimal 64 is encoded as one byte 0x40; decimal 321 is encoded as two bytes 0xC1 0x02. The maximum remaining length is 256 MiB, and MQTT does not support packet fragmentation, making it unsuitable for very large payloads such as video streams.

Variable Header

The variable header sits between the fixed header and the payload. Different control packets have different variable headers. The Packet Identifier (PacketId) is a two‑byte field present in many packets, including PUBLISH (QoS > 0), PUBACK, PUBREC, PUBREL, PUBCOMP, SUBSCRIBE, SUBACK, UNSUBSCRIBE, and UNSUBACK.

Notes

PUBLISH QoS 0 packets must not contain a PacketId.

For some control packets the PacketId must match the corresponding packet (e.g., PUBLISH QoS 1 ↔ PUBACK, PUBLISH QoS 2 ↔ PUBCOMP).

Senders and receivers maintain their own PacketId spaces, so duplicate identifiers may appear on opposite sides.

Packets without a PacketId cannot reuse the identifier because there is no acknowledgment to indicate when it can be reclaimed.

Payload

The payload is the application message. MQTT 3.1.1 defines the following control packets: CONNECT, CONNACK, PUBLISH, PUBACK, PUBREC, PUBREL, PUBCOMP, SUBSCRIBE, SUBACK, UNSUBSCRIBE, UNSUBACK, PINGREQ, PINGRESP, and DISCONNECT. Their specific formats and use cases will be explained in future articles.

Conclusion

MQTT is a binary protocol with bit‑level control fields, which gives it a strong position in the IoT field. However, it does not support packet fragmentation and is therefore not suitable for scenarios with very large data packets, such as video streaming.