Why Choose RocketMQ Over Kafka? The Real Reasons Behind the 90% Mistake

Interview Scenario

An interview asks why Kafka can achieve 1‑2 million TPS while RocketMQ tops out at 300‑600 k TPS, and which one to choose for a 10 billion‑message daily e‑commerce system.

Design Philosophy

Kafka treats messages as immutable logs and focuses on raw I/O efficiency – zero‑copy, sequential writes, and batch processing. RocketMQ is built for critical business events, supporting transactions, delayed delivery, ordering, and tag filtering.

Storage Model Comparison

Kafka : each partition is a plain log file; messages are appended directly, and offsets serve as identifiers.

RocketMQ : uses a CommitLog plus per‑queue ConsumeQueue index files. Writing involves two I/O steps – message body to CommitLog and index entry to ConsumeQueue.

I/O Layer Differences

Kafka leverages zero‑copy (sendfile) for both producer writes and consumer reads, avoiding user‑space copies. RocketMQ must copy data from kernel buffers to user‑space to parse topics, tags, and other metadata before writing, resulting in two explicit CPU memory copies.

Batching and Compression

Kafka batches messages (default 16 KB) and supports snappy/gzip/lz4 compression, then uses zero‑copy to send batches. RocketMQ also supports batching and compression but defaults to smaller batch sizes and requires manual configuration, limiting throughput.

Horizontal Scaling

Kafka partitions are independent and can be reassigned across brokers, enabling seamless scaling.

RocketMQ queues are fixed at topic creation; adding brokers does not automatically rebalance existing queues, so scaling is less flexible.

Performance Numbers

6‑node Kafka cluster: 800 k‑1.2 M TPS (pure produce/consume).

6‑node RocketMQ cluster: 300‑600 k TPS under the same hardware.

Why Large Companies Still Prefer RocketMQ

For e‑commerce core flows, features outweigh raw throughput:

Scenario 1 – Transactional Order Creation

// Send half‑message (transactional)
TransactionSendResult result = producer.sendMessageInTransaction(msg, orderId);
try {
    redisService.decrStock(orderId); // local stock deduction
    return LocalTransactionState.COMMIT_MESSAGE;
} catch (Exception e) {
    return LocalTransactionState.ROLLBACK_MESSAGE;
}

Kafka lacks native transaction messages, forcing developers to implement complex compensation logic.

Scenario 2 – Delayed Order Cancellation

Message message = new Message("t_order_timeout", "close_order", body);
message.setDelayTimeLevel(7); // 30 minutes
producer.send(message);

RocketMQ provides 18 built‑in delay levels; Kafka requires external schedulers.

Scenario 3 – Ordered Processing

// Send to the same queue based on order ID
MessageQueue mq = queueList.get(orderId.hashCode() % queueNum);
producer.send(message, mq);

// Consumer registers ordered listener
consumer.registerMessageListener(new ConsumeOrderlyListener() {
    @Override
    public ConsumeOrderlyStatus consumeMessage(List<MessageExt> msgs, ConsumeOrderlyContext ctx) {
        // process in order
        return ConsumeOrderlyStatus.SUCCESS;
    }
});

Kafka can achieve ordering with partition keys, but RocketMQ’s API is more straightforward for strict ordering.

Absolute Question 1 – CPU Copies in RocketMQ Write Path

Two major copies occur:

Kernel network buffer → Netty DirectByteBuf (user‑space copy).

User‑space buffer → mmap‑mapped CommitLog region (memcpy).

Additional tiny copies happen when writing index entries to ConsumeQueue.

Absolute Question 2 – Heap vs Off‑Heap

Core message data flows entirely in off‑heap memory (Netty DirectByteBuf → mmap). Only lightweight metadata (topic strings, tags) are allocated on the Java heap.

Absolute Question 3 – Does CommitLog Write Touch the Heap?

By default, no. The message body is copied directly from off‑heap buffers to the mmap region; only minimal metadata resides on the heap.

Performance Optimizations for RocketMQ

Producer: enable batch sending, set compressMsgBodyOverHowmuch, increase batchMaxSize.

Broker: use ASYNC_FLUSH, enlarge writeBufferSize, increase mapedFileSizeCommitLog.

Increase topic queue count (e.g., from 8 to 32) to raise parallel consumption.

Deploy additional master/slave brokers for load distribution.

System‑Level Buffering Solution

Introduce an off‑heap ring buffer (Disruptor) at the API gateway to absorb traffic spikes, batch messages, and push them to RocketMQ at a controlled rate, achieving peak QPS of 2 w with <80 ms P99 latency.

RingBuffer<MessageEvent> ringBuffer = RingBuffer.create(
    ProducerType.MULTI,
    MessageEvent::new,
    65536,
    new YieldingWaitStrategy()
);

Scheduled task drains the buffer every 20 ms or when 512 messages accumulate and sends the batch to RocketMQ.

scheduler.scheduleAtFixedRate(() -> {
    List<Message> batch = buffer.drainTo(512);
    if (!batch.isEmpty()) {
        try { producer.send(batch); }
        catch (Exception e) { fallbackStorage.saveToLocalDB(batch); }
    }
}, 0, 20, TimeUnit.MILLISECONDS);

Final Takeaway

Kafka excels at raw throughput for log‑oriented pipelines, while RocketMQ sacrifices some speed to provide transactional guarantees, delayed delivery, ordered consumption, and richer filtering – essential for financial‑grade e‑commerce systems. The right choice depends on business semantics, not just TPS numbers.