Why Kafka Marks a Live Consumer as Dead and Forces Rebalance

During high‑traffic releases or peak periods a consumer may appear healthy—its JVM is alive, logs are printed, and business code continues processing—but Kafka server declares the consumer dead and forces a rebalance. This condition is called 假死 (false dead).

Why a heartbeat alone is not enough

Early Kafka versions used a single consumer thread that both called poll() to fetch records and performed the heartbeat. If business logic blocked the thread (e.g., a database timeout or a Full GC pause), the heartbeat could not be sent. The broker, not receiving a heartbeat within session.timeout.ms (default 45 s), assumed the client had crashed and immediately triggered a rebalance.

Dual‑thread architecture

Modern Kafka separates these responsibilities into two threads:

Heartbeat thread – continuously sends heartbeats; the broker considers the client alive as long as heartbeats arrive within session.timeout.ms.

Business thread – repeatedly calls poll() to fetch records and execute business logic. The broker monitors the interval between successive poll() calls using max.poll.interval.ms (default 5 min). If this interval exceeds the limit, the broker treats the consumer as unable to make progress, marks it as false dead, and initiates a rebalance.

Example: a consumer pulls 500 messages in one batch. If processing those 500 messages takes 6 minutes, the business thread cannot return to poll() before the 5‑minute deadline. Although the heartbeat thread continues sending heartbeats, the broker sees no recent poll() and decides the consumer is dead, kicking it out of the group.

How to avoid false dead

Reduce batch size : set max.poll.records to a smaller value (e.g., 50). Smaller batches keep processing time per poll() short, allowing the thread to return to the poll loop before the interval expires.

Increase the poll interval : raise max.poll.interval.ms (e.g., to 600000 ms) to accommodate worst‑case processing scenarios, but adjust max.poll.records accordingly; an excessively large interval can delay detection of a truly stuck consumer.

Asynchronous multithreaded consumption : let the main thread only poll and immediately hand the records to a custom thread pool for parallel processing. This keeps the poll loop fast, but introduces offset‑commit risks:

With automatic commits, the next poll() may commit offsets for records still being processed, causing message loss if the process crashes.

With manual commits, offsets may be committed out of order, leading to duplicate consumption unless a sophisticated sliding‑window or per‑partition queue mechanism is implemented.

Final thoughts

Distributed systems define liveness in two ways: process liveness (the process is up, ports open, heartbeats alive) and business liveness (the application can still process incoming data). When writing Kafka consumer logic, you must consider both max.poll.records and max.poll.interval.ms to ensure the broker perceives the consumer as truly alive and to avoid unnecessary rebalances.