Tagged articles
Idempotence
41 articles · Page 1 of 1
The article analyzes the requirements of a massive‑scale like feature, compares small‑scale MySQL implementations with the graph‑plus‑KV architectures used by ByteDance, Kuaishou and Xiaohongshu, and presents a practical hybrid solution based on Redis, MySQL and batch processing to handle hot content and super‑node challenges.
IdempotenceKV storeMySQL
0 likes · 19 min read

The article explains why fixed‑interval retries cause thousands of producers to synchronize their retries and overload brokers, and how exponential backoff, full jitter, retry budgeting, and differentiated policies per topic can safely handle send failures even at tens of millions of QPS.
ExponentialBackoffHighThroughputIdempotence
0 likes · 26 min read

The article explains why RocketMQ inevitably delivers duplicate messages under at‑least‑once semantics, analyzes root causes in producer, broker and consumer stages, and presents a production‑grade idempotent solution that combines business keys, Redis caching, Redisson locks, a MySQL idempotent table, AOP interception, and comprehensive monitoring to guarantee exactly‑once business outcomes even under high concurrency and Kubernetes graceful shutdown.
IdempotenceJavaRedis
0 likes · 34 min read

The article explains why Kafka does not guarantee exactly‑once delivery and presents three practical approaches—business‑level idempotence, manual offset management, and Kafka’s transaction/EOS features—to reliably avoid duplicate message processing.
ConsumerIdempotenceKafka
0 likes · 4 min read

The article explains why merely “knowing how to use” Kafka is insufficient, detailing how offset commits, consumer acknowledgments, producer acks, and rebalance behavior affect reliability, and provides concrete code examples, risk scenarios, and configuration recommendations to prevent message loss and duplicate processing in production systems.
IdempotenceKafkaMessage Reliability
0 likes · 7 min read

This article demystifies Kafka's ordering guarantees, explains the difference between partition‑level, key‑level, and global ordering, and provides a complete, production‑grade guide—including architecture, configuration, code samples, and best‑practice patterns—to achieve reliable ordered processing while maintaining high throughput and scalability.
CDCConsumerIdempotence
0 likes · 35 min read

This article explains Kafka’s core mechanisms for preventing data loss, covering replica architecture, leader‑follower roles, ACK policies with ISR, log persistence and flush strategies, as well as producer idempotence and transactional support that together ensure reliable, exactly‑once delivery.
ACKIdempotenceKafka
0 likes · 6 min read

This guide explains how to avoid duplicate message consumption in Kafka by designing unique identifiers, implementing consumer-side idempotence with deduplication tables, leveraging Kafka’s transactional features, and establishing system-level safeguards and monitoring to ensure reliable, exactly‑once processing.
IdempotenceKafkaMessage Queue
0 likes · 4 min read

This article explains why services must be gracefully shut down before leveraging Spring Event, how startup timing can cause event loss, which consistency scenarios suit or reject publish‑subscribe, and practical reliability techniques such as retries, idempotence, and fault reporting.
Event‑DrivenIdempotenceJava
0 likes · 11 min read

This guide explains how to configure Kafka producers with acks=all, enable idempotence, set appropriate retry and replication factors, and adjust broker flush intervals to achieve exactly‑once semantics and prevent message loss even during network glitches or broker failures.
IdempotenceKafkaMessage Reliability
0 likes · 4 min read

This article explains Kafka's idempotent producer feature, describing how enabling idempotence ensures that duplicate messages caused by retries or network issues are deduplicated so that each message is persisted only once in a partition, and outlines the underlying PID and sequence number mechanism.
IdempotenceMessage Queueexactly-once
0 likes · 4 min read

This guide explains how to efficiently produce messages with TDMQ CKafka, covering topic creation, partition sizing, retry policies, asynchronous sending, key/value handling, batch tuning, sticky partitioning, idempotence, ACK settings, code examples, and strategies to avoid data skew and ensure high‑throughput, low‑latency messaging.
CKafkaIdempotenceKafka
0 likes · 18 min read

This article explains the three message delivery semantics, the common causes of duplicate messages in queue systems, and presents concrete producer‑side, broker‑side, and consumer‑side deduplication techniques for Kafka and Pulsar, including code samples and best‑practice recommendations.
BackendIdempotenceKafka
0 likes · 9 min read

This article explains common causes of duplicate message consumption in high‑traffic systems and presents a three‑layer defense—producer idempotence, broker de‑duplication, and consumer idempotent design—plus monitoring and reconciliation strategies to achieve reliable exactly‑once processing.
IdempotenceKafkaMessage Queue
0 likes · 7 min read

Message queues can deliver duplicates under at‑least‑once semantics, so to protect idempotent business logic such as orders and payments you should combine producer‑side idempotence (e.g., Kafka’s enable.idempotence), broker‑side deduplication (e.g., Pulsar), and a consumer‑side guard using unique IDs stored in a database or Redis.
IdempotenceKafkaMessage Queue
0 likes · 7 min read

Message queues can cause duplicate messages that affect idempotent business processes, so this article explains the three delivery semantics (At Least Once, Exactly Once, At Most Once), the causes of duplication, and practical deduplication techniques for producers, brokers (Kafka, Pulsar), and consumers using code examples.
IdempotenceKafkaPulsar
0 likes · 8 min read

This article shares hard‑won lessons from production incidents and provides practical guidelines—graceful shutdown, proper startup timing, suitable business scenarios, reliability patterns, and idempotent handling—to use Spring Event safely and effectively in Java backend systems.
BackendEventIdempotence
0 likes · 12 min read

This article explains the concept of idempotence, why it matters in distributed systems, how to handle timeout scenarios, design principles using globally unique IDs, and eight practical implementation patterns—including database tricks, token schemes, locking, and HTTP method considerations—to build robust, repeat‑safe APIs.
BackendHTTPIdempotence
0 likes · 21 min read

The article explains the concept of idempotence in computing, illustrates typical duplicate‑request scenarios, and presents three practical backend solutions—unique database indexes, optimistic locking with version control, and a Redis‑based token mechanism—along with their advantages, trade‑offs, and implementation details.
BackendHTTPIdempotence
0 likes · 7 min read

Idempotence ensures that repeating the same operation yields the same result without side effects, a critical property in distributed and transactional systems; the article defines the concept, explains why it matters, lists common causes of non‑idempotent behavior, and presents several practical backend solutions such as unique indexes, anti‑duplicate tables, optimistic and pessimistic locks, tokens, distributed locks, and state machines.
BackendIdempotencedistributed systems
0 likes · 8 min read

This article explains why message duplication occurs in Kafka, describes the three delivery semantics, and provides practical solutions—including idempotent producers, transactions, and consumer-side idempotence—along with configuration tips and code examples to achieve exactly‑once delivery.
ConfigurationIdempotenceJava
0 likes · 11 min read

This article explains how Kafka achieves producer idempotence by assigning unique producer IDs and sequence numbers, describes the broker’s validation process, and walks through the relevant producer‑side and broker‑side code paths, highlighting configuration considerations and limitations.
IdempotenceKafkaMessage Queue
0 likes · 13 min read

This article explains Kafka's internal topics, preferred replicas, partition assignment processes, log directory layout, index files, offset and timestamp lookup, log retention and compaction policies, storage architecture, delayed operations, controller role, legacy consumer design flaws, rebalance workflow, and producer idempotence, providing a comprehensive overview of Kafka's backend architecture.
Consumer OffsetsIdempotenceLog Retention
0 likes · 16 min read

This article explains why message middleware guarantees at‑least‑once delivery, the problems caused by duplicate consumption, and presents both transaction‑based and non‑transactional deduplication solutions—including a Redis‑backed idempotent table—to achieve exactly‑once semantics in RocketMQ.
IdempotenceRocketMQdistributed systems
0 likes · 19 min read

This article analyzes the at‑least‑once guarantee of message middleware, explains why duplicate deliveries occur, compares simple database‑based deduplication methods, explores concurrency challenges, and presents a non‑transactional, status‑driven idempotence solution using MySQL or Redis with practical code examples and limitations.
IdempotenceMySQLRedis
0 likes · 19 min read

This article explains why message middleware guarantees at‑least‑once delivery, describes three common duplication scenarios in RocketMQ, and presents both transactional and non‑transactional deduplication solutions—including SQL examples and a Redis‑based idempotence library—to achieve exactly‑once processing.
IdempotenceRocketMQdistributed systems
0 likes · 19 min read

Message middleware guarantees at-least-once delivery, but repeated deliveries cause duplicate processing; this article explains RocketMQ's duplicate scenarios, explores simple and advanced deduplication techniques—including database-transactional and Redis-based idempotent tables—and provides practical Java code for implementing exactly-once semantics.
IdempotenceRocketMQdistributed systems
0 likes · 20 min read

This article explains why message middleware guarantees at‑least‑once delivery, the challenges of duplicate consumption, and presents both transactional and non‑transactional deduplication strategies—including database and Redis‑based solutions—to achieve idempotent processing in RocketMQ.
IdempotenceMessage QueueRocketMQ
0 likes · 19 min read

This article explains why message middleware guarantees at‑least‑once delivery, the challenges of duplicate consumption, and presents both simple and advanced deduplication strategies—including transactional and non‑transactional approaches using relational databases or Redis—to achieve effectively exactly‑once semantics in RocketMQ.
IdempotenceRedisRocketMQ
0 likes · 18 min read

This article explains common scenarios that cause message loss or non‑idempotent processing in RabbitMQ and Kafka, and presents practical solutions such as persistent delivery, confirm mechanisms, delayed delivery, and unique‑ID plus fingerprint strategies to achieve reliable and idempotent message transmission.
IdempotenceKafkaRabbitMQ
0 likes · 6 min read

This article explains Kafka’s three delivery semantics—At most once, At least once, and Exactly once—from both producer and consumer perspectives, details the required configuration settings, and discusses how Kafka ensures idempotence, transaction support, and prevents data loss, duplication, and ordering issues.
Delivery GuaranteesIdempotenceKafka
0 likes · 23 min read

This article explains Kafka's exactly‑once semantics and transaction support, detailing the new producer API methods, related exceptions, configuration parameters, and a sample application illustrating how to initialize, begin, process, and commit or abort transactions while ensuring idempotent and atomic message handling.
ConfigurationIdempotenceKafka
0 likes · 19 min read

This article explains how to build a high‑availability service layer for large‑scale web systems by vertically splitting business domains, adopting stateless design, configuring timeouts, using asynchronous messaging, ensuring idempotent operations, and applying service degradation techniques to maintain stability during traffic spikes.
BackendIdempotenceService Layer
0 likes · 11 min read

This article provides an in‑depth overview of Apache Kafka’s internal topics, preferred replicas, partition allocation mechanisms, log directory structure, index files, offset and timestamp lookup, log retention and compaction policies, storage architecture, delayed operations, controller role, consumer rebalance process, and producer idempotence.
Consumer RebalanceIdempotenceKafka
0 likes · 18 min read

The article explains Kafka's reliability mechanisms, detailing how committed messages are persisted, common producer and consumer data‑loss scenarios, best‑practice configurations for acks, retries, replication, and offset handling, and describes idempotent and transactional producer setups for atomic writes.
IdempotenceKafkaMessage Reliability
0 likes · 7 min read

This article reviews typical Kafka interview topics, explaining the implementation of delay queues with hierarchical time wheels, how idempotence is achieved via producer IDs, the meanings of ISR, OSR, AR, HW, LEO, LSO, strategies for guaranteeing message order, and practical solutions for handling duplicate consumption.
ISRIdempotencedelay queue
0 likes · 9 min read

This article explains common scenarios where Kafka messages can be lost on the producer, consumer, or broker side, and provides practical configurations—including callbacks, acks, retries, manual offset commits, idempotent and transactional producers—to ensure reliable delivery and exactly‑once processing.
IdempotenceMessage LossTransactions
0 likes · 10 min read

This article reviews common Kafka interview questions, covering delay queues, idempotence, replica states, offsets, message ordering, and handling duplicate consumption, and includes example code for enabling idempotent producers along with explanations of time‑wheel mechanisms and practical solutions to consumer rebalance issues.
ConsumerIdempotenceInterview
0 likes · 9 min read

This article explains Kafka's Exactly-Once Semantics (EOS), the role of idempotence, and how transactional support works, covering EOS semantics, producer id and sequence numbers, configuration properties, and providing Java code examples for initializing, beginning, committing, and aborting transactions.
IdempotenceJavaKafka
0 likes · 8 min read

This article explains how Apache Kafka implements producer idempotence by introducing Producer IDs (PID) and sequence numbers, describes the request‑response flow for PID allocation, details server‑side PID management, shows the exact‑once guarantee mechanism, and answers common configuration questions with code examples.
BackendIdempotenceKafka
0 likes · 32 min read

This article explains the concept of idempotence, provides real‑world examples such as duplicate form submissions and payment requests, and details practical backend techniques—including query handling, unique indexes, token mechanisms, pessimistic and optimistic locks, distributed locks, and API design—to guarantee that repeated operations produce consistent results without side effects.
Idempotencedatabasedistributed-systems
0 likes · 10 min read
