Designing a Message Queue: Key Considerations and Architecture

Message queues have become a core communication mechanism in enterprise IT systems, offering low coupling, reliable delivery, broadcasting, flow control, and eventual consistency, and serving as a primary means for asynchronous RPC.

The article first discusses scenarios where a message queue is needed, such as decoupling services, achieving eventual consistency, broadcasting updates, and handling traffic spikes, contrasting it with direct RPC when strong consistency is required.

It then outlines the design of a message queue, starting with the overall data flow (producer → broker → consumer) and the two main responsibilities: message persistence for later delivery and providing a common paradigm for decoupling, consistency, and traffic shaping.

Key components covered include the RPC communication protocol, high‑availability strategies (leveraging existing RPC frameworks and shared storage for idempotency), and the choice of storage subsystem (in‑memory, distributed KV, file system, or database) based on performance and reliability requirements.

The handling of consumer relationships, broadcast versus unicast, and the maintenance of subscription information in systems like ZooKeeper or config servers are explained.

Advanced features such as reliable delivery, transaction support, and dealing with duplicate or out‑of‑order messages are discussed, with approaches like message logging, compensation tasks, version numbers, and state‑machine validation.

Performance considerations cover asynchronous versus synchronous RPC, batch processing, and the trade‑offs between push and pull models, including strategies for slow consumers, flow control, and message ordering.

Code examples illustrate client‑side asynchronous calls and batch sending using executors and blocking queues:

Future<Result> future = request(server); // server returns future immediately
synchronized (future) {
    while (!future.isDone()) {
        future.wait(); // server notifies when done
    }
}
return future.get();

Result result = request(server);

Future<Result> future = executor.submit(new Callable(){
    public Result call(){
        return request(server);
    }
});
return future;

Future<Result> future = request(server); // server returns future immediately
return future;

Executor executor = Executors.newFixedThreadPool(4);
final BlockingQueue<Message> queue = new ArrayBlockingQueue<>();
private Runnable task = new Runnable(){
    public void run(){
        List<Message> messages = new ArrayList<>(20);
        queue.drainTo(messages, 20);
        doSend(messages); // block, new messages may arrive
    }
};
public void send(Message message){
    queue.offer(message);
    executor.submit(task);
}

Executor executor = Executors.newFixedThreadPool(4);
final BlockingQueue<Message> queue = new ArrayBlockingQueue<>();
volatile long last = System.currentTimeMillis();
Executors.newSingleThreadScheduledExecutor().submit(new Runnable(){
    public void run(){ flush(); }
}, 500, 500, TimeUnit.MILLISECONDS);
private Runnable task = new Runnable(){
    public void run(){
        List<Message> messages = new ArrayList<>(20);
        queue.drainTo(messages, 20);
        doSend(messages);
    }
};
public void send(Message message){
    last = System.currentTimeMillis();
    queue.offer(message);
    flush();
}
private void flush(){
    if (queue.size() > 200 || System.currentTimeMillis() - last > 200) {
        executor.submit(task);
    }
}

In conclusion, the article provides a comprehensive roadmap for building a custom message queue, highlighting essential design decisions and pointing to future topics such as storage subsystem design, flow control, and fair scheduling.