Message Queue Interview Guide: Applications, Frameworks, Design Patterns, and Best Practices

Hello, I am Tom. With the upcoming holiday, I wish everyone a happy break.

To help readers quickly grasp the full picture of message queues and typical interview questions, I have compiled a structured guide.

Message queue application scenarios

1. Asynchronous processing – decouple non‑core steps to reduce response time, e.g., sending SMS after user registration. 2. Traffic shaping – smooth spikes such as flash‑sale events. 3. Application decoupling – isolate services so a failure in one does not affect the other, e.g., async inventory deduction. 4. Message communication – point‑to‑point or chat‑room messaging.

Common message queue frameworks

ActiveMQ, RabbitMQ, ZeroMQ, Kafka, MetaQ, RocketMQ, Pulsar, etc.

MQ technology selection

Comparison of Kafka, RocketMQ, and Pulsar based on latency, throughput, reliability, transaction support, replica sync strategy, multi‑tenant capability, dynamic scaling, and fault recovery.

Message models

1. Point‑to‑point (queue) model. 2. Publish/subscribe (topic) model.

Ensuring no message loss

Producer side – guarantee persistence.

Broker side – reliable storage.

Consumer side – acknowledge processing.

For detailed mechanisms, see the referenced articles.

Handling duplicate consumption

Producers may retry sending until ACK, causing duplicates; mature brokers deduplicate via message‑id tracking and provide idempotent send APIs. Consumers must implement idempotency, e.g., using a database flag or Redis cache.

Guaranteeing message order

1. Global order – single partition per topic (reduces concurrency). 2. Partial order – route related messages (e.g., same order) to the same partition using a partition key; consume single‑threaded. Kafka offers org.apache.kafka.clients.Partitioner for custom routing.

Dealing with message backlog

Scale consumer group instances to increase throughput.

Investigate slow consumer processing (e.g., missing DB index causing 500 ms per message).

Data consistency via asynchronous messaging

Perform local DB transaction first, then emit an MQ message; the consumer completes downstream actions. Use transactional messages to ensure atomicity, and apply eventual consistency with retry mechanisms.

How transactional messages work

Producer sends a half‑transaction message.

Broker acknowledges receipt.

Producer executes local transaction.

If successful, send commit; otherwise, send rollback.

If broker does not receive final status, it initiates a reverse check.

Producer queries final transaction state.

Based on the query, it sends the appropriate final confirmation.

Achieving high throughput in MQ frameworks

Batch processing of messages.

Message compression.

Zero‑copy I/O.

Sequential disk writes.

Page‑cache usage for asynchronous flushing.

Partitioned design allowing parallel consumption.

Why Kafka does not support read/write separation

Both producers and consumers interact with the leader replica to maintain consistency and low latency; follower replicas lag behind, making separate read replicas impractical.

High availability in MQ frameworks (Kafka example)

Kafka clusters consist of multiple brokers; topics are split into partitions with replicated replicas. Leaders handle reads/writes, followers replicate data. If a broker fails, a follower is elected leader, ensuring continuous service.

Typical Kafka interview questions

Message compression and decompression.

Partitioning strategies.

Producer idempotence and transactions.

Broker storage and backup mechanisms.

Purpose of consumer groups.

For more details, refer to the linked articles.