Why Kafka Prioritizes High Availability Over Read/Write Separation

Background

Working daily as a CRUD developer, the author rarely encounters high‑performance, high‑concurrency, high‑availability scenarios, so they explore how popular open‑source middleware such as Redis and Kafka achieve these goals.

Why does Kafka need high availability?

Two main reasons:

Hardware failures become normal – As cluster size grows, machine outages are common, requiring a failover mechanism to keep the service running.

Risk of data loss – Before version 0.8 Kafka lacked HA; a broker crash could lose messages, which forces Kafka to provide HA to protect critical data.

How Kafka implements high availability

Typical middleware HA solutions involve four key processes:

Redundancy

Failure detection

Leader election

External notification

For detailed design, see the referenced summary (link omitted).

Why doesn’t Kafka offer read/write separation?

Although partition replication provides redundancy, Kafka does not expose a read/write‑separate mode for several reasons:

Message consumption is not pure read

Consuming a message involves a write operation: the consumer commits the offset, recording the position of the last successfully processed message.

while(true){
    consumer.poll(); //① pull messages
    // process message
    consumer.commit(); //② commit offset (write)
}

Master‑slave data sync simplicity

If read/write separation were supported, followers would need to sync offset data back to the leader, creating an n×n synchronization topology and increasing complexity and error risk. A one‑way sync (leader writes, followers read) remains simpler.

Timeliness of message consumption

Read/write separation introduces latency because reads from followers may lag behind writes to the leader. This delay affects the freshness of consumed data, which is undesirable for many real‑time use cases.

Summary

Kafka implements high availability to address hardware failure normalcy and data‑loss risk. Its core HA ideas are redundancy, failure detection, leader election, and external notification. Kafka does not provide read/write separation because consumption involves writes, master‑slave sync is simpler without it, and read latency would degrade consumption timeliness.

Link: https://juejin.cn/post/7340295754244882441