An Overview of the MQTT Protocol: History, Requirements, and Message Format

Recently I have been studying the MQTT protocol and compiled a summary of its history, the problems it aims to solve, and the basic format of its messages. The content is originally sourced from a technical discussion article.

MQTT is a lightweight, hub‑centric messaging system that enables communication between sensors, applications, and devices via a central broker. Its low‑bandwidth nature makes it suitable for many IoT scenarios such as home automation, HVAC, lighting, smart devices, and security.

1. History

MQTT was invented in 1999 by Andy Stanford‑Clark (IBM) and Arlen Nipper (Arcom, now Cirrus Link) to minimize battery consumption and bandwidth when connecting oil‑pipeline monitoring stations over satellite links. The protocol was defined with the following requirements:

Simple implementation

Quality‑of‑service aware data transmission

Lightweight and bandwidth‑efficient

Stateless data handling

Persistent session awareness

2. Protocol Format

MQTT control packets consist of three parts: a fixed header, an optional variable header, and a payload. The fixed header is present in every packet, while the variable header and payload appear only in certain packet types.

Fixed Header

The fixed header is two bytes long. The first byte’s high‑order bits (7‑4) indicate the control packet type, and the low‑order bits (3‑0) contain flags specific to that packet type. The second byte encodes the remaining length of the packet using a variable‑length scheme that can occupy up to four bytes.

Variable Header

The variable header sits between the fixed header and the payload. Its content varies by packet type; for example, many packets include a two‑byte Packet Identifier (PacketId) used in PUBLISH (QoS > 0), PUBACK, PUBREC, PUBREL, PUBCOMP, SUBSCRIBE, SUBACK, UNSUBSCRIBE, and UNSUBACK.

Payload

The payload carries the application message. In MQTT 3.1.1 the protocol defines control packets such as CONNECT, CONNACK, PUBLISH, PUBACK, PUBREC, PUBREL, PUBCOMP, SUBSCRIBE, SUBACK, UNSUBSCRIBE, UNSUBACK, PINGREQ, PINGRESP, and DISCONNECT. Detailed formats and use‑cases for each will be covered in future articles.

Summary

MQTT is a binary protocol with bit‑level precision in its control fields, which makes it well‑suited for many IoT applications. However, because it does not support packet fragmentation, it is not ideal for scenarios involving very large data transfers, such as video streaming.