What Makes Disque a Fault‑Tolerant Distributed Message Broker?

Disque is a new open‑source distributed in‑memory message broker authored by Salvatore Sanfilippo, the original creator of Redis. It targets the “Redis as a job queue” use case with a dedicated, independent, scalable, fault‑tolerant design, combining Redis’s simplicity and high performance while being implemented in C as a non‑blocking network server. In the documentation and this article, the terms “message” and “job” are used interchangeably.

Although Disque is a separate project from Redis, it reuses much of Redis’s networking code, node bus, libraries, and client protocol. Because it shares Redis’s protocol, any Redis client can connect to a Disque cluster directly; the default port is 7711 instead of 6379.

As a message broker, Disque sits between producer and consumer processes, allowing producers to add messages that consumers later retrieve. The common producer‑consumer queue model is retained, with several distinctive details:

Disque is a synchronous‑replication job queue; by default a new job is replicated to W nodes, and the failure of W‑1 nodes does not affect delivery.

It supports both at‑least‑once and at‑most‑once delivery semantics. The former is the primary design focus; the latter can be achieved by setting the retry time to zero. Each message’s delivery semantics are set individually, allowing mixed semantics within the same queue.

At‑least‑once delivery is “approximate once”; Disque strives to avoid duplicate deliveries.

All nodes have the same role in a multi‑master cluster. Producers and consumers may connect to different nodes, and the cluster automatically exchanges messages based on load and client requests.

Optional asynchronous commands let a producer add a job to a queue with a replication factor greater than one without waiting for replication to finish; replication proceeds in the background.

When a configured retry timeout expires, Disque automatically re‑queues messages that have not received a response.

Consumers acknowledge messages explicitly to signal successful processing.

Disque provides best‑effort ordering based on the creation timestamp from the local node’s clock. Messages created on the same node are usually delivered in creation order, but strict FIFO guarantees are not provided, especially after re‑queuing or load‑balanced moves. Consequently, Salvatore notes that Disque is technically a message broker rather than a strict queue.

Four parameters give fine‑grained job control: replication factor (number of copies), delay (minimum wait before the message enters the queue), retry time (when the message is re‑queued), and TTL (when the message expires).

Disque is currently in an Alpha preview; its code and algorithms have not been fully tested and it is not recommended for production environments. In the coming months the implementation and API are expected to change significantly. Additional limitations include:

Many unused Redis code fragments remain.

The project originated from observing Redis being used as a queue, not from a specific internal need, and the author is not an expert in queue systems.

Like Redis, Disque is single‑threaded, though its simple data structures could allow a future multithreaded version.

The number of jobs a Disque process can handle is bounded by available memory.

No performance optimizations have been applied.

In short, Disque remains a research‑testing project. Interested readers can visit its GitHub repository for the latest information.