BestHub
Sign in
Tagged articles
13 articles
Page 1 of 1
Architect Chen
Architect Chen
Mar 3, 2026 · Backend Development

Preventing Kafka Duplicate Consumption with Idempotent Design

This article explains practical strategies to avoid duplicate message consumption in Kafka, covering business idempotency with unique IDs, database or Redis deduplication tables, enabling producer idempotence, consumer-side checks, and Kafka's transaction-based exactly‑once semantics, along with their trade‑offs and suitable scenarios.

BackendExactly-OnceIdempotency
0 likes · 4 min read
Preventing Kafka Duplicate Consumption with Idempotent Design
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Jul 14, 2025 · Backend Development

Kafka Producer Idempotency: PID, Sequence Numbers, and Broker Deduplication

Kafka ensures that a producer’s repeated message sends, caused by network glitches or broker failures, result in only one persisted record per partition by using a unique Producer ID, monotonically increasing sequence numbers, and broker-side tracking of the latest committed sequence for each PID‑partition pair.

BackendKafkaMessage Deduplication
0 likes · 5 min read
Kafka Producer Idempotency: PID, Sequence Numbers, and Broker Deduplication
Xuanwu Backend Tech Stack
Xuanwu Backend Tech Stack
May 7, 2025 · Backend Development

How to Prevent Duplicate Message Consumption in RabbitMQ

RabbitMQ often suffers from duplicate message consumption due to network glitches, node failures, or ACK issues, but you can eliminate repeats by implementing idempotent processing, using caches or deduplication tables, and leveraging manual acknowledgments or the Message Deduplication plugin, as illustrated with Java examples.

JavaManual AcknowledgmentMessage Deduplication
0 likes · 6 min read
How to Prevent Duplicate Message Consumption in RabbitMQ
Top Architect
Top Architect
Oct 24, 2021 · Backend Development

Message Deduplication and Exactly‑Once Semantics in RocketMQ

This article explains why RocketMQ guarantees at‑least‑once delivery, describes the three typical duplicate‑message scenarios, compares transaction‑based and non‑transactional deduplication approaches (including a Redis‑based solution), provides sample SQL and Java code, and discusses the limitations and best‑practice recommendations for achieving idempotent message consumption.

Exactly-OnceIdempotencyJava
0 likes · 20 min read
Message Deduplication and Exactly‑Once Semantics in RocketMQ
Java Backend Technology
Java Backend Technology
Sep 20, 2021 · Backend Development

How to Achieve Exactly‑Once Message Processing in RocketMQ Without Transactions

This article explains why message middleware guarantees at‑least‑once delivery, the problems caused by duplicate consumption, and presents both transaction‑based and non‑transactional deduplication solutions—including a Redis‑backed idempotent table—to achieve exactly‑once semantics in RocketMQ.

Distributed SystemsExactly-OnceIdempotence
0 likes · 19 min read
How to Achieve Exactly‑Once Message Processing in RocketMQ Without Transactions
Java Architect Essentials
Java Architect Essentials
Sep 17, 2021 · Backend Development

How to Achieve Exactly‑Once Message Processing in RocketMQ Without Transactions

This article analyzes the at‑least‑once guarantee of message middleware, explains why duplicate deliveries occur, compares simple database‑based deduplication methods, explores concurrency challenges, and presents a non‑transactional, status‑driven idempotence solution using MySQL or Redis with practical code examples and limitations.

Distributed SystemsExactly-OnceIdempotence
0 likes · 19 min read
How to Achieve Exactly‑Once Message Processing in RocketMQ Without Transactions
Architecture Digest
Architecture Digest
Sep 17, 2021 · Backend Development

Message Deduplication and Exactly-Once Semantics in RocketMQ

This article explains why message middleware guarantees at‑least‑once delivery, describes three common duplication scenarios in RocketMQ, and presents both transactional and non‑transactional deduplication solutions—including SQL examples and a Redis‑based idempotence library—to achieve exactly‑once processing.

Distributed SystemsExactly-OnceIdempotence
0 likes · 19 min read
Message Deduplication and Exactly-Once Semantics in RocketMQ
Java Interview Crash Guide
Java Interview Crash Guide
Aug 30, 2021 · Backend Development

How to Achieve Exactly-Once Message Processing with RocketMQ Deduplication

Message middleware guarantees at-least-once delivery, but repeated deliveries cause duplicate processing; this article explains RocketMQ's duplicate scenarios, explores simple and advanced deduplication techniques—including database-transactional and Redis-based idempotent tables—and provides practical Java code for implementing exactly-once semantics.

Distributed SystemsExactly-OnceIdempotence
0 likes · 20 min read
How to Achieve Exactly-Once Message Processing with RocketMQ Deduplication
Programmer DD
Programmer DD
Jul 22, 2021 · Backend Development

How to Achieve Exactly‑Once Message Processing in RocketMQ Without Heavy Transactions

This article explains why message middleware guarantees at‑least‑once delivery, the challenges of duplicate consumption, and presents both simple and advanced deduplication strategies—including transactional and non‑transactional approaches using relational databases or Redis—to achieve effectively exactly‑once semantics in RocketMQ.

Exactly-OnceIdempotenceMessage Deduplication
0 likes · 18 min read
How to Achieve Exactly‑Once Message Processing in RocketMQ Without Heavy Transactions
21CTO
21CTO
Nov 10, 2019 · Mobile Development

How We Built a Scalable Mobile IM System for a Travel App

This article details the design and implementation of a scalable mobile instant‑messaging system for a travel app, covering architecture, data‑channel handling, subscription mechanisms, UI rendering, message de‑duplication, local push strategies, reconnection logic, and future upgrades to WebSocket.

Instant MessagingMessage DeduplicationMobile
0 likes · 16 min read
How We Built a Scalable Mobile IM System for a Travel App
JD Retail Technology
JD Retail Technology
Oct 25, 2019 · Backend Development

Design and Optimization Strategies for High‑Concurrency Instant Messaging Processing

This article explains how to handle high‑concurrency instant‑messaging scenarios by designing message deduplication, optimizing SQLite write performance with batch inserts and WAL mode, improving query speed with indexes and caching, and reducing UI refresh overhead through delayed and scroll‑aware updates.

Instant MessagingMessage DeduplicationPerformance Optimization
0 likes · 19 min read
Design and Optimization Strategies for High‑Concurrency Instant Messaging Processing
Mafengwo Technology
Mafengwo Technology
Oct 17, 2019 · Mobile Development

How We Built a Scalable Mobile IM System for Travel Apps

This article details the design, architecture, and implementation of a mobile instant‑messaging system for a travel platform, covering data channel design, message subscription, UI rendering, deduplication, local push, reconnection strategies, and the planned migration to WebSocket.

Message DeduplicationWebSocketpush notifications
0 likes · 17 min read
How We Built a Scalable Mobile IM System for Travel Apps