BestHub
Sign in
Tagged articles
6 articles
Page 1 of 1
Su San Talks Tech
Su San Talks Tech
Jan 30, 2026 · Backend Development

Why Kafka Rebalance Causes Backlog, Duplicates, and Data Loss—and How to Fix It

Kafka consumer group rebalances can trigger message backlogs, duplicate processing, and data loss; this article explains common rebalance triggers, their impact on consumption, and practical configuration and coding strategies—such as tuning timeout parameters, using manual offset commits, and sticky partition assignment—to minimize disruptions.

Data lossKafkaMessage Duplication
0 likes · 12 min read
Why Kafka Rebalance Causes Backlog, Duplicates, and Data Loss—and How to Fix It
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Dec 27, 2025 · Backend Development

Why Kafka Messages Duplicate and How to Prevent It

The article explains the main causes of duplicate Kafka messages—including producer retries, consumer offset handling, partition leader changes, and lack of idempotence—and provides practical configuration and design solutions to achieve exactly‑once delivery.

Consumer OffsetMessage DuplicationReplication
0 likes · 5 min read
Why Kafka Messages Duplicate and How to Prevent It
Java Tech Enthusiast
Java Tech Enthusiast
Apr 30, 2024 · Backend Development

How to Prevent Message Loss and Duplication in Message Queues

This article examines why messages can be lost or duplicated in typical queue systems, explains the failure points from producer to broker to consumer, and provides practical techniques such as synchronous flushing, broker clustering, database unique constraints, and Redis deduplication to achieve reliable, idempotent processing.

BackendIdempotencyKafka
0 likes · 7 min read
How to Prevent Message Loss and Duplication in Message Queues
Top Architect
Top Architect
Mar 30, 2023 · Backend Development

Understanding Kafka Idempotent Producer and How to Prevent Message Duplicates

This article explains why message duplication occurs in Kafka, describes the three delivery semantics, and provides practical solutions—including idempotent producers, transactions, and consumer-side idempotence—along with configuration tips and code examples to achieve exactly‑once delivery.

ConfigurationIdempotenceJava
0 likes · 11 min read
Understanding Kafka Idempotent Producer and How to Prevent Message Duplicates
dbaplus Community
dbaplus Community
Mar 20, 2017 · Backend Development

How RocketMQ Tackles Message Ordering, Duplication, and Transaction Challenges

This article explains how RocketMQ, a high‑throughput distributed messaging middleware, solves the classic problems of message ordering, duplicate delivery, and transactional consistency, while detailing its load‑balancing producer, storage architecture, subscription models, and practical best‑practice recommendations.

Backend DevelopmentMessage DuplicationMessage Ordering
0 likes · 23 min read
How RocketMQ Tackles Message Ordering, Duplication, and Transaction Challenges