Designing a Hundred‑Billion‑Scale Message Queue: A ByteDance Interview Walkthrough

Problem Statement

Design a message‑queue system that processes 100 billion messages per day (≈10 TB of inbound data) with a daily average QPS of 115 k, a 4‑hour peak of 700 k, and a possible burst of up to 1 million QPS. Retention of 7 days requires roughly 70 TB of disk capacity.

Core Roles

Producer : Sends messages to the broker.

Broker : Temporarily stores messages and provides routing, storage, and high‑availability services.

Consumer : Pulls or receives messages for processing.

Registry : Handles service discovery, broker health monitoring, and routing metadata.

Core Functions of a Message Queue

Decoupling : Asynchronous messaging removes direct API calls between upstream and downstream services.

Asynchrony : Non‑core logic (e.g., SMS, points update) is off‑loaded to the queue, improving the main flow’s response time.

Peak‑shaving : Messages are buffered during traffic spikes, allowing downstream services to consume at their own pace.

Architecture Design Dimensions

High‑Throughput Storage

Use sequential disk I/O (append‑only CommitLog) to avoid random‑write penalties.

Apply zero‑copy techniques such as Java FileChannel.transferTo() combined with batch pulls to reduce I/O overhead.

Tiered Storage for Massive Data

Hot data resides on local SSD for low‑latency reads/writes.

Cold data is asynchronously off‑loaded to inexpensive object storage (e.g., S3, MinIO).

Pulsar’s compute‑storage separation (BookKeeper) natively supports tiered storage.

Scalability

Fixed hash sharding (Kafka, RocketMQ): Route messages by hashing a key (user ID, order ID) modulo the number of partitions; guarantees ordering within a partition.

Consistent hashing : Map physical nodes onto a hash ring; adding or removing nodes only moves data on adjacent nodes, improving re‑balancing efficiency.

Kafka’s partitions increase throughput linearly as the partition count grows.

High Availability & Reliability

Kafka ISR (In‑Sync Replicas) with acks=all ensures that all ISR replicas acknowledge a write before the producer receives success.

RocketMQ offers synchronous or asynchronous replication; synchronous disk flush + synchronous double‑write is common for hundred‑billion‑scale scenarios to achieve zero‑loss guarantees.

Observability

Producer metrics: send latency, success rate, failure reasons.

Broker metrics: request throughput, backlog size, disk usage, I/O latency.

Consumer metrics: consumption lag, offset, retry count.

MQ Framework Comparison (key specs)

RabbitMQ – Erlang; complex routing, low latency; ~100 k QPS; strong backlog; single‑queue ordering; push/pull consumer model.

Kafka – Scala/Java; high‑throughput log processing; ~200 k QPS; very strong backlog; ordering per partition; pull consumer model.

RocketMQ – Java; financial‑grade reliability; ~100 k QPS; strong backlog; ordering per partition; pull consumer model.

Pulsar – Java; cloud‑native, compute‑storage separation; ~1 M QPS; very strong backlog with tiered storage; ordering per partition; pull consumer model.

Real‑World Case Study

Original Bottlenecks

Explosion of long‑lived producer‑broker connections.

Large volume of tiny messages causing excessive network I/O and disk writes.

Cross‑region synchronization latency > 200 ms.

Post‑Refactor Design

Cluster‑Group abstraction : Logical topics map to physical clusters; dynamic routing and metadata sync reduced cross‑region latency from 200 ms to 35 ms and cut operational effort by 80 %.

Prism proxy layer : Consolidated broker connections, lowering connection count by 92 % and boosting per‑node throughput from 120 k msg/s to 2.8 M msg/s. Techniques included batch aggregation, connection reuse, gRPC streaming, and intelligent retry.

Multi‑region disaster‑recovery : Deployed across multiple availability zones, used cross‑region replicas, automatic failover, and an improved ISR‑like consistency guarantee.

Full‑stack monitoring with Prometheus + Grafana and automated operations ensured continuous availability.

Frequent Interview Questions

How to guarantee message order? RocketMQ requires sending all messages of an ordered stream to the same MessageQueue and consuming them with a single thread; normal ordering allows parallel consumption across different queues.

How to ensure no message loss? Producer: synchronous send + retry; Broker: synchronous flush + double‑write; Consumer: manual ACK + idempotent handling.

How to handle duplicate consumption? Implement idempotency on the consumer side (e.g., unique DB keys or state‑machine checks).

How to deal with backlog? Temporarily scale consumer instances, optimize slow SQL, and optionally discard non‑critical messages with compensation.

Conclusion

Identify the four core roles (Producer, Broker, Consumer, Registry) and the data flow Producer → Broker → Consumer.

Explain three design dimensions: high‑throughput storage (sequential write + zero‑copy), scalability (hash sharding + consistent hashing), and high availability (replication + ISR).

Illustrate with a real‑world case that shows batch aggregation, a proxy layer, and multi‑region design to achieve low latency and high throughput.

Select the appropriate MQ based on scenario: Kafka for massive log throughput, RocketMQ for financial‑grade reliability, Pulsar for cloud‑native elasticity.