Mastering Python’s struct Module: Binary Data Packing and Unpacking Explained

1. Introduction

The struct module solves conversion between bytes and other binary data types in Python, making stream operations simple and addressing the lack of a native byte data type.

2. Uses

1. Convert Python data to a byte‑stream string according to a specified format, e.g., for network transmission where integers must be sent as bytes.

2. Convert a byte‑stream back to specified Python data types.

3. Handle binary files; when using struct with files, open them in binary mode ('wb', 'rb').

3. Functions in struct

See the table below:

4. Format strings

When packing or unpacking, a format string defines data types and includes special characters to control byte order, size, and alignment.

5. Alignment

To exchange data with C structs, consider the compiler’s byte alignment (typically 4‑byte on 32‑bit systems). The first character of the format string can change alignment.

Endianness: big‑endian stores the most significant byte at the lowest address; little‑endian stores it at the highest address.

6. Format characters

See the table below:

Notes:

_Bool is defined in C99; if unavailable, it is treated as a char (1 byte).

q and Q are for 64‑bit machines only.

A number preceding a format indicates repetition, e.g., 4s for a 4‑character string, 4i for four integers.

P converts a pointer; its size depends on the machine.

f and d sizes depend on the machine.

7. Examples

Typical packing: bytes = pack(...)

Unpacking may overflow if the buffer length exceeds the expected size; add a length check.

For image data, read the file first then unpack.

pack_into places various data objects into a buffer and can define the packing range; unpack_from does the reverse.

calcsize returns the memory size of a format string.

8. Summary

This article introduced the Python struct module for handling binary streams, covering its purpose, functions, format strings, alignment, format characters, and common usage examples.