Tagged articles

manual commit

3 articles · Page 1 of 1

Aug 8, 2026 · Backend Development

Why Does Kafka Re‑Consume Tens of Thousands of Messages After a Successful Offset Commit?

The article explains why Kafka consumers can reprocess tens of thousands of messages after a restart despite successful offset commits, detailing the roles of automatic commit timing, rebalance triggers, GC pauses, and the limitations of manual commits, and offers practical configuration and idempotent processing solutions.

ConsumerJavaKafka

0 likes · 8 min read

Why Does Kafka Re‑Consume Tens of Thousands of Messages After a Successful Offset Commit?

IT Services Circle

Aug 5, 2026 · Backend Development

Why Kafka Offsets That Appear Committed Still Cause Duplicate Consumption After Restart

The article explains why a Kafka consumer can re‑process tens of thousands of messages after a restart even though the offset was reported as successfully committed, covering auto‑commit timing, rebalance triggers, GC pauses, manual‑commit pitfalls, and practical mitigation strategies.

KafkaRebalanceauto-commit

0 likes · 8 min read

Why Kafka Offsets That Appear Committed Still Cause Duplicate Consumption After Restart

Random Bulletin

Jun 15, 2026 · Backend Development

From Auto to Manual: Mastering Consumer Offsets for Ten‑Million QPS Systems

The article examines a real payment loss incident caused by Kafka's default automatic offset commit, explains why automatic commits become a hidden trap at high traffic, and provides a step‑by‑step guide to switching to manual commits with async‑first, sync‑fallback, idempotency, dead‑letter handling, batch strategies, and concurrency controls for reliable ten‑million QPS consumption.

IdempotencyKafkadead-letter queue

0 likes · 15 min read

From Auto to Manual: Mastering Consumer Offsets for Ten‑Million QPS Systems