Inside RabbitMQ Classic Queue: How Its Storage Architecture Impacts Performance

Classic Queue Overview

RabbitMQ is a widely used open‑source message broker. In its architecture, virtual hosts (vhosts) isolate resources, exchanges handle routing, and queues are the smallest unit of message storage. Classic queues are the most common type, balancing lightweight indexing with shared storage to suit high‑throughput, non‑strong‑consistency scenarios.

Storage Architecture

Each vhost has its own physical directory. The typical layout is:

vhost_name/
├── msg_store_persistent/   # shared storage for large messages
│   ├── 0.rdq              # shared storage file
│   └── 1.rdq              # supports file rolling
└── queues/
    └── queue_name/
        ├── queue_name.qi # index file (metadata)
        └── queue_name.qs # storage file (message payloads)

The msg_store_* directory is shared by all queues in the vhost; it stores messages that exceed a size threshold to avoid duplicate copies. Each queue also has its own sub‑directory under queues containing two key files.

Queue Index and Storage Files

The index file ( *.qi) holds a header and a series of entries that map message IDs to their location in either the queue‑specific storage file ( *.qs) or the shared storage. Two entry types exist: Publish Entry (created when a producer sends a message) and Ack Entry (overwrites the publish entry after the consumer acknowledges the message). Important fields include:

MsgId – a GUID generated by RabbitMQ that identifies the message in shared storage.

SeqId – the sequence number of the message within the queue, used to locate the entry in both index and storage files.

The storage file ( *.qs) mirrors the index file structure: a header followed by entries that contain the actual payload for messages whose size is below the queue_index_embed_msgs_below threshold (default 4 KB).

Core Write Process

When a producer publishes a message, RabbitMQ decides based on size whether to write to the queue‑specific storage or the shared storage. The message is first placed in an in‑memory buffer (512 KB for queue storage, 1 MB for shared storage). Once the buffer exceeds its limit, the data is flushed to the operating system’s page cache without an fsync, favoring latency over durability. After the flush, a Publish Entry is appended to the index file, and the in‑memory cache is updated to accelerate subsequent reads.

Core Read Process

Consumers read messages from an in‑memory cache that is periodically refilled with up to 2 048 messages from disk. If a requested message is not in the cache, RabbitMQ reads the corresponding Publish Entry from the index, then fetches the payload from either the queue storage or the shared storage, assembles the full message, and returns it. Large messages (>12 KB) or very small batches (<10 messages) are read directly from disk rather than being cached.

File Compaction

Shared‑storage files are compacted when the ratio of valid data falls below 50 %. Compaction proceeds in three steps: (1) move valid data from the file’s end to fill gaps left by deleted messages, (2) update the Index component with the new locations, and (3) truncate the file after no process is reading it. This process creates “blank holes” that are filled only if a later message fits; otherwise the file size remains unchanged to avoid overwriting moved data.

Operational Recommendations

Enable publisher confirms so the broker acknowledges a message only after it has been flushed to disk, providing stronger delivery guarantees.

Use manual acknowledgments (manual ack) on the consumer side; RabbitMQ will only delete a message after its Ack Entry replaces the Publish Entry, ensuring no loss if a consumer crashes.

Set an appropriate prefetch count to limit the number of unacknowledged messages held in memory, preventing OOM situations.

Keep queues short by matching production and consumption rates; RabbitMQ automatically removes index and storage files once all messages are consumed, freeing disk space and improving read performance.

Conclusion

The classic queue’s storage subsystem combines a lightweight index with shared and per‑queue storage files, using configurable buffers, entry counts, and compaction thresholds to balance performance, durability, and disk usage. Understanding these mechanisms helps operators tune RabbitMQ for high‑throughput workloads and troubleshoot storage‑related issues.