Interview Basics: How to Guarantee Message Reliability in MQ

Ensuring that a message is produced, persisted, and consumed exactly once is a classic interview topic for message‑queue (MQ) middleware. The answer must address three dimensions: producer 100% delivery, MQ persistence, and consumer 100% consumption (no loss, no duplication).

RabbitMQ

RabbitMQ follows the AMQP protocol. Its core components are Exchange, Queue, and Binding, which together form the processing chain.

Producer side : Enable confirm mode on the channel. Each published message receives a unique ID. If the broker accepts the message, it sends an ack via ConfirmCallback; if routing fails, a nack with error details is sent via ReturnCallback. Only one of ack or nack is triggered, and the callbacks are asynchronous.

Consumer side : Declare the queue with noAck=false so the broker waits for a manual ack before removing the message. If the consumer crashes, the broker re‑delivers the message, so consumer code must be idempotent.

Persistence :

Exchange durability – set durable=true in channel.exchangeDeclare(..., true).

Queue durability – set durable=true in

channel.queueDeclare("queue.persistent.name", true, false, false, null)

.

Message durability – publish with MessageProperties.PERSISTENT_TEXT_PLAIN or set deliveryMode=2 via AMQP.BasicProperties.Builder().deliveryMode(2).

When durability flags are enabled, RabbitMQ appends messages to log files (default 16 MB segment size) and rotates files automatically. An ETS table tracks file offsets. Garbage collection merges or deletes segments when the garbage ratio exceeds 50 % and at least three files exist. Disk buffers are flushed every 25 ms (or immediately if no further writes occur).

Kafka

Kafka is designed for high throughput and large message backlogs. Its architecture includes Consumer Groups, Topics, Partitions, and Replicas (Leader + Followers). Data is written to disk sequentially, leveraging zero‑copy I/O, segment files with accompanying index files, and batch compression.

Producer side : acks=0 – no acknowledgment, high loss risk. acks=1 – leader writes successfully before ack. acks=all (or -1) – all in‑sync replicas must acknowledge.

Set unclean.leader.election.enable=false to avoid electing out‑of‑ISR followers as leaders, improving reliability at the cost of availability.

Configure retries>1 for message resend.

Set min.insync.replicas>1 so writes are rejected unless the required number of replicas are in sync.

Consumer side : Disable automatic offset commits; manually commit the offset only after successful processing, ensuring the ability to re‑consume on failure. Consumer code must also be idempotent.

Broker persistence :

Sequential disk reads/writes avoid random seeks.

Zero‑copy transfers bypass user‑space copying.

Messages are stored in segment files; each segment has an index file for fast lookup.

Batch compression and batch I/O reduce bandwidth usage.

Kafka writes directly to the OS page cache, not the JVM heap, eliminating GC pauses.

Flush policies can be tuned with three parameters: log.flush.interval.messages – number of messages per flush. log.flush.interval.ms – time interval per flush. log.flush.scheduler.interval.ms – periodic scheduler interval.

Both RabbitMQ and Kafka provide mechanisms to balance performance and reliability. Understanding these underlying protocols, configuration knobs, and persistence strategies is essential for answering interview questions on MQ reliability.