Skip to content
Tagged articles

Data Consistency

212 articles · Page 2 of 3
IT Architects Alliance
IT Architects Alliance
Sep 6, 2022 · Operations

How to Guarantee Zero Message Loss with Kafka: Best Practices and Configurations

This article explains why introducing a message queue like Kafka helps decouple systems and control traffic, then dives into the three key questions of detecting, locating, and preventing message loss, offering concrete monitoring methods, configuration settings, and troubleshooting steps for producers, brokers, and consumers.

ConsumerData ConsistencyKafka
0 likes · 13 min read
How to Guarantee Zero Message Loss with Kafka: Best Practices and Configurations
DataFunTalk
DataFunTalk
Jun 25, 2022 · Big Data

Alluxio Metadata and Data Synchronization: Design, Implementation, and Optimization

This article provides a comprehensive overview of Alluxio's metadata and data synchronization mechanisms, covering its unified namespace, mounting strategies, consistency models, various write modes, read workflows, metadata sync techniques, performance optimizations, and recommended configurations for different deployment scenarios.

AlluxioData ConsistencyStorage
0 likes · 26 min read
Alluxio Metadata and Data Synchronization: Design, Implementation, and Optimization
Shopee Tech Team
Shopee Tech Team
Mar 17, 2022 · Backend Development

Real-time Checking System for Data Consistency in Microservices

Shopee’s Real‑time Checking System provides configurable, non‑intrusive data consistency verification for micro‑services by capturing change events via CDC, streaming them through Kafka, applying flexible rules and expressions, and instantly alerting mismatches, delivering second‑level detection while scaling to tens of thousands of checks per second.

CDCData ConsistencyDistributed Systems
0 likes · 20 min read
Real-time Checking System for Data Consistency in Microservices
Selected Java Interview Questions
Selected Java Interview Questions
Mar 1, 2022 · Backend Development

Cache Consistency Strategies: From Simple Write‑Through to Binlog Subscription

This article explains why caching is essential for high‑concurrency systems, analyzes the challenges of keeping database and cache data consistent, and compares five practical cache‑invalidation strategies—including write‑after‑DB, delete‑before‑write, delayed double delete, queue‑based deletion, and binlog subscription—highlighting their trade‑offs and suitable scenarios.

BackendCacheData Consistency
0 likes · 10 min read
Cache Consistency Strategies: From Simple Write‑Through to Binlog Subscription
IT Architects Alliance
IT Architects Alliance
Feb 27, 2022 · Backend Development

Why Message Queues Matter: Decoupling, Asynchrony, and Real‑World Pitfalls

This article explains how message queues help decouple services, enable asynchronous processing, smooth traffic spikes, and improve system resilience, while also detailing common challenges such as reduced availability, increased complexity, duplicate consumption, data consistency, message loss, ordering, and backlog, along with practical mitigation strategies.

Backend ArchitectureData ConsistencyDecoupling
0 likes · 15 min read
Why Message Queues Matter: Decoupling, Asynchrony, and Real‑World Pitfalls
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Feb 21, 2022 · Backend Development

How JD.com Scales Its E‑Commerce Platform: Vision, Principles, and Governance

After a decade of rapid growth, JD.com’s information system now handles procurement, sales, and inventory across a horizontally and vertically split architecture, tackling challenges of service decoupling, efficient communication, data consistency, and governance, while outlining its vision, principles, implementation details, and stability‑enhancing governance practices.

Data ConsistencyService Governancee-commerce
0 likes · 3 min read
How JD.com Scales Its E‑Commerce Platform: Vision, Principles, and Governance
iQIYI Technical Product Team
iQIYI Technical Product Team
Feb 11, 2022 · Databases

Design and Implementation of iQIYI Content Platform Data Center (OLTP)

The paper describes iQIYI’s Content Platform Data Center (OLTP), a unified data hub that solves microservice data‑island problems by providing hundred‑billion‑scale storage, high‑QPS reads/writes, field‑level change notifications, final consistency between MongoDB and Elasticsearch, active‑active high availability, and a generic SDK, now serving 26 business lines with thousands of QPS and hundreds of millions of rows.

Data ConsistencyMongoDBOLTP
0 likes · 11 min read
Design and Implementation of iQIYI Content Platform Data Center (OLTP)
NiuNiu MaTe
NiuNiu MaTe
Feb 9, 2022 · Backend Development

How to Resolve Account Transaction Conflicts with Split Writes and Lazy Integration

This article explains the "account conflict" problem caused by massive concurrent transactions, proposes splitting read/write operations and using lazy or timed data integration to improve performance and consistency, and discusses how to ensure atomicity and isolation with transactions or distributed locks.

Data Consistencydistributed-lock
0 likes · 6 min read
How to Resolve Account Transaction Conflicts with Split Writes and Lazy Integration
Aikesheng Open Source Community
Aikesheng Open Source Community
Jan 25, 2022 · Databases

Custom MySQL Data Consistency Check Tool: Design, Implementation, and Usage

This article introduces a custom MySQL data consistency verification tool inspired by pt-table-checksum, explains the challenges of handling complex primary keys, details the implementation logic with shell scripts and SQL, and provides installation, configuration, and usage instructions including parallelism, network monitoring, and scheduling.

Data Consistencyautomationchecksum
0 likes · 23 min read
Custom MySQL Data Consistency Check Tool: Design, Implementation, and Usage
Cloud Native Technology Community
Cloud Native Technology Community
Jan 12, 2022 · Cloud Native

Choosing Cloud‑Native Persistent Storage Solutions for Financial Container Platforms

The article examines how banks can select and implement cloud‑native persistent storage on Kubernetes‑based container platforms, covering storage types, CSI integration, DBaaS options, data consistency challenges, and strategies for high‑concurrency fault recovery in the financial sector.

DBaaSData ConsistencyFinancial Services
0 likes · 8 min read
Choosing Cloud‑Native Persistent Storage Solutions for Financial Container Platforms
Dada Group Technology
Dada Group Technology
Dec 10, 2021 · Operations

Design and Practice of the Freight Business Check System (BCS)

The article introduces the freight BCS system, explains its business background, describes multiple validation modes for data consistency and business logic correctness, compares implementation approaches, and outlines the architecture, task flow, and future enhancements to improve system reliability and operational monitoring.

BackendData Consistencybusiness validation
0 likes · 10 min read
Design and Practice of the Freight Business Check System (BCS)
vivo Internet Technology
vivo Internet Technology
Nov 3, 2021 · Backend Development

Evolution and Architecture of vivo Mall's Product System

From its 2017 v2.0 upgrade to a service‑oriented design, vivo Mall’s product system was split from the monolithic mall, progressively adding independent activity, flash‑sale, consignment, and inventory services while employing rate limiting, multi‑level caching, circuit breaking, and distributed transactions to ensure stability, high performance, and data consistency.

Data Consistencybackend developmentcaching strategies
0 likes · 9 min read
Evolution and Architecture of vivo Mall's Product System
High Availability Architecture
High Availability Architecture
Oct 16, 2021 · Backend Development

Ensuring Data Consistency Between MySQL and Redis: Strategies for Single‑Threaded and Multi‑Threaded Scenarios

This article explains what data consistency means for MySQL and Redis, analyzes inconsistency cases in both single‑threaded and concurrent environments, and proposes practical strategies—including read‑only and read‑write cache handling, message‑queue retries, binlog subscription, delayed double‑delete, and distributed locking—to achieve eventual or strong consistency.

Data ConsistencyDistributed LocksMessage Queue
0 likes · 16 min read
Ensuring Data Consistency Between MySQL and Redis: Strategies for Single‑Threaded and Multi‑Threaded Scenarios
Wukong Talks Architecture
Wukong Talks Architecture
Oct 14, 2021 · Backend Development

Understanding Redis and MySQL Data Consistency and Caching Strategies

This article explains when and how to use caching with Redis, compares recursive and table‑lookup algorithms for factorial calculation, discusses cache design, expiration policies, and presents several write‑through and write‑behind strategies to maintain data consistency between Redis and MySQL under various concurrency scenarios.

Data Consistencymysql
0 likes · 12 min read
Understanding Redis and MySQL Data Consistency and Caching Strategies
Tencent Cloud Developer
Tencent Cloud Developer
Oct 13, 2021 · Databases

Ensuring Data Consistency Between MySQL and Redis: Strategies and Best Practices

To keep MySQL and Redis synchronized, the article defines consistency, examines read‑only and read‑write cache inconsistency cases, and recommends practical strategies such as update‑DB‑then‑delete‑cache with message‑queue retries, binlog‑driven updates, delayed double‑delete, and Redis distributed locks, while outlining stronger protocols and key design considerations.

Data Consistencycache strategydistributed-lock
0 likes · 15 min read
Ensuring Data Consistency Between MySQL and Redis: Strategies and Best Practices
dbaplus Community
dbaplus Community
Sep 26, 2021 · Databases

Why MySQL Auto‑Increment Can Leak Data and How Distributed IDs Offer a Safer Alternative

The article examines the 2012 GitHub data‑leak caused by MySQL master‑slave failover and auto_increment misuse, explains why developers' expectations of uniqueness, monotonicity and continuity are unrealistic, and proposes half‑sync replication and distributed ID algorithms like Snowflake as more reliable solutions.

Data ConsistencyDatabase ReplicationSnowflake
0 likes · 9 min read
Why MySQL Auto‑Increment Can Leak Data and How Distributed IDs Offer a Safer Alternative
Architect's Journey
Architect's Journey
Sep 3, 2021 · Backend Development

Five Critical Questions to Test Your System Design’s Reasonableness

The article outlines five essential dimensions—data consistency, isolation, extensibility, business sequencing, and high‑concurrency handling—explaining why each matters, illustrating them with concrete examples, and showing how to evaluate and improve a technical design accordingly.

Architecture DesignCAP theoremData Consistency
0 likes · 10 min read
Five Critical Questions to Test Your System Design’s Reasonableness
Architect's Tech Stack
Architect's Tech Stack
Aug 27, 2021 · Backend Development

Transaction Management Patterns in Microservices: Blocking Retry, Async Queue, TCC, and Local Message Table

The article explains common microservice transaction patterns—including blocking retry, asynchronous queues, TCC compensation transactions, and local message tables—detailing their implementations, advantages, drawbacks, and practical code examples for ensuring data consistency in distributed systems.

Data ConsistencyDistributed SystemsTCC
0 likes · 14 min read
Transaction Management Patterns in Microservices: Blocking Retry, Async Queue, TCC, and Local Message Table
ITPUB
ITPUB
Aug 23, 2021 · Backend Development

Ensuring Microservice Data Consistency: Retry, Queues, TCC & Message Tables

The article examines common strategies for handling service call failures and maintaining data consistency in microservice architectures, comparing blocking retries, asynchronous queues, TCC compensation transactions, local message tables, and MQ transactions, highlighting their trade‑offs, implementation details, and practical considerations.

Data ConsistencyTCCdistributed transactions
0 likes · 16 min read
Ensuring Microservice Data Consistency: Retry, Queues, TCC & Message Tables
Sohu Tech Products
Sohu Tech Products
Aug 4, 2021 · Backend Development

Resolving Duplicate OpenID Insertions with Distributed Locks in a Fast App Center

To prevent duplicate OpenID records caused by concurrent synchronization requests in the Fast App Center, this article analyzes the root cause, evaluates database‑level unique indexes versus application‑level distributed locks, and presents a Redis‑based lock implementation with cleanup procedures to ensure data consistency.

BackendData ConsistencyJava
0 likes · 16 min read
Resolving Duplicate OpenID Insertions with Distributed Locks in a Fast App Center
Code Ape Tech Column
Code Ape Tech Column
Jul 13, 2021 · Backend Development

Ensuring Consistent Money Transfers: Distributed Transactions & Message Queues Explained

The article examines how to prevent data inconsistency during cross‑service money transfers by using local transactions, two‑phase commit, and message‑queue based eventual consistency, providing detailed code examples, performance considerations, and practical solutions for large‑scale backend systems.

BackendData ConsistencyMessage Queue
0 likes · 23 min read
Ensuring Consistent Money Transfers: Distributed Transactions & Message Queues Explained
Xianyu Technology
Xianyu Technology
Jul 9, 2021 · Backend Development

Backend Architecture and Stability for Xianyu Local Services

The article describes Xianyu’s local services architecture, tackling rapid supplier onboarding, heterogeneous quality, and stability by reusing core platform capabilities, defining merchant, audit, and independent business domains, employing high‑concurrency rate limiting, idempotent retries, unified exception handling, status‑change logging, and proactive monitoring with alerts and reporting.

Data Consistencymonitoringstability
0 likes · 7 min read
Backend Architecture and Stability for Xianyu Local Services
Code Ape Tech Column
Code Ape Tech Column
Jul 7, 2021 · Backend Development

How to Ensure Data Consistency in Microservices: From Blocking Retries to TCC

This article examines common techniques for handling service call failures in micro‑service architectures—blocking retries, asynchronous queues, TCC compensation transactions, local message tables, and MQ transactions—detailing their implementations, pitfalls, and trade‑offs to achieve reliable data consistency.

Data ConsistencyMessage QueueTCC
0 likes · 16 min read
How to Ensure Data Consistency in Microservices: From Blocking Retries to TCC
DeWu Technology
DeWu Technology
Jun 25, 2021 · Information Security

DCheck: Real-time Asset Loss Detection and Prevention System

DCheck is a real‑time asset‑loss detection platform that monitors MySQL binlog and MQ streams, matches upstream and downstream states, and triggers alerts via configurable topics, events, sub‑events and Groovy scripts implementing filter and check logic, while offering rule‑based controls, debugging tips, and proposed enhancements for reliability.

Asset Loss PreventionData ConsistencyGroovy Scripts
0 likes · 11 min read
DCheck: Real-time Asset Loss Detection and Prevention System
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Jun 18, 2021 · Fundamentals

What Makes Distributed File Systems Tick? Design Principles and Architecture Explained

This article examines the core concepts, requirements, architectural models, persistence strategies, scalability, high‑availability mechanisms, performance optimizations, security models, and practical considerations of distributed file systems such as HDFS, GFS, and Ceph, offering a comprehensive guide for engineers and researchers.

Data ConsistencyHigh Availabilitydistributed file system
0 likes · 21 min read
What Makes Distributed File Systems Tick? Design Principles and Architecture Explained
IT Architects Alliance
IT Architects Alliance
Jun 17, 2021 · Backend Development

A General Cache Handling Mechanism for Static Business Data in Microservice Architecture

The article proposes a comprehensive microservice‑based caching solution for low‑frequency static data such as vehicle models and user profiles, detailing why caching is needed, why Redis and persistent queues are chosen, how consistency checks work, and the trade‑offs compared with simple expiration strategies.

Backend ArchitectureData ConsistencyQueue
0 likes · 14 min read
A General Cache Handling Mechanism for Static Business Data in Microservice Architecture
Wukong Talks Architecture
Wukong Talks Architecture
Jun 17, 2021 · Databases

Ensuring Data Consistency Without Native Transactions in MongoDB

The article explains how lack of native transactions in MongoDB 3.0 can cause data inconsistency during order processing, compares it with SQL transaction mechanisms, and proposes optimization and compensation strategies such as retry queues, asynchronous tasks, and refund handling to ensure eventual consistency.

Data ConsistencyMongoDBTransaction
0 likes · 10 min read
Ensuring Data Consistency Without Native Transactions in MongoDB
21CTO
21CTO
Jun 16, 2021 · Backend Development

How to Build a Universal Static Data Cache for Microservices with Redis

This article explains a reusable caching architecture for low‑frequency static data in microservice systems, covering why caching is needed, the role of Redis, persistent queues, consistency checks, and trade‑offs such as cache expiration and operational complexity.

Data ConsistencyQueuecaching
0 likes · 14 min read
How to Build a Universal Static Data Cache for Microservices with Redis
Su San Talks Tech
Su San Talks Tech
Jun 7, 2021 · Backend Development

Why Region IDs Break OpenAPI Integrations and How to Design Resilient Solutions

This article examines the challenges of synchronizing enterprise data across systems when region identifiers differ, explores several technical approaches—including persisting local tables, name‑based and code‑based lookups, snapshot storage, manual updates, and flexible API fields—and concludes with a pragmatic, business‑centric solution for reliable integration.

API designBackend IntegrationData Consistency
0 likes · 14 min read
Why Region IDs Break OpenAPI Integrations and How to Design Resilient Solutions
Wukong Talks Architecture
Wukong Talks Architecture
Mar 19, 2021 · Fundamentals

Understanding Zookeeper: Architecture, Nodes, Sessions, Watchers, Leader Election, and Consistency

This article provides a comprehensive overview of Zookeeper, covering its purpose, cluster roles, Znode types, session handling, watcher mechanism, ACL permissions, common use cases, data consistency via the ZAB protocol, leader election, synchronization methods, potential inconsistency scenarios, and a comparison with other service‑registry solutions.

ConsensusData ConsistencyLeader Election
0 likes · 16 min read
Understanding Zookeeper: Architecture, Nodes, Sessions, Watchers, Leader Election, and Consistency
JD Retail Technology
JD Retail Technology
Mar 12, 2021 · Backend Development

Cache Synchronization in High‑Concurrency Environments: Problems and JD's CDC‑Based Solution

The article reviews common cache‑side data‑sync patterns, highlights their inconsistency and data‑loss risks under high load, and presents JD's solution that combines Cache‑Aside, Change Data Capture, message queues, delayed consumption, versioning, and persistence to ensure eventual consistency between cache and relational databases.

CDCCacheData Consistency
0 likes · 7 min read
Cache Synchronization in High‑Concurrency Environments: Problems and JD's CDC‑Based Solution
Architects' Tech Alliance
Architects' Tech Alliance
Mar 1, 2021 · Cloud Computing

Practices for Data Cleaning and Cutover Consistency in Cross‑Cloud Migration

This article explains the technical details of data cleaning, dirty‑data handling, and three methods—database read‑only, application termination, and network ACL isolation—to ensure data consistency during the data‑regulation and cutover phases of cross‑cloud migration, illustrated with real‑world case studies.

ACL isolationCloud MigrationData Consistency
0 likes · 12 min read
Practices for Data Cleaning and Cutover Consistency in Cross‑Cloud Migration
dbaplus Community
dbaplus Community
Feb 8, 2021 · Backend Development

Avoid Stale Data: Pitfalls and Best Practices for Cache Aside, Read‑Through, Write‑Through, and Write‑Behind

This article explains why cache‑database inconsistencies occur in large systems, details the cache‑aside, read‑through, write‑through and write‑behind strategies, highlights three common pitfalls with concrete examples, and offers practical recommendations such as proper update ordering and cache expiration to ensure data freshness.

BackendCacheCache Aside
0 likes · 9 min read
Avoid Stale Data: Pitfalls and Best Practices for Cache Aside, Read‑Through, Write‑Through, and Write‑Behind
Architect's Journey
Architect's Journey
Jan 26, 2021 · Backend Development

Three Storage Solutions for Cross-Database Aggregated Full-Text Search

The article compares three approaches—synchronous dual write, asynchronous dual write with a message queue, and CDC via Canal—to keep Elasticsearch and a relational database consistent for cross‑database aggregated full‑text search, outlining their steps, advantages, and drawbacks.

Backend ArchitectureCDCData Consistency
0 likes · 6 min read
Three Storage Solutions for Cross-Database Aggregated Full-Text Search
Top Architect
Top Architect
Jan 4, 2021 · Backend Development

Understanding the Saga Pattern for Distributed Data Consistency in Microservices

This article explains the importance of data consistency in distributed microservice systems, introduces the Saga pattern as a solution, compares it with two‑phase commit and TCC, and details its architecture, recovery mechanisms, and implementation considerations within ServiceComb.

Data ConsistencyTCCTwo-Phase Commit
0 likes · 19 min read
Understanding the Saga Pattern for Distributed Data Consistency in Microservices
Architect
Architect
Dec 31, 2020 · Backend Development

Understanding the Saga Pattern for Distributed Data Consistency in Microservices

This article explains why data consistency is critical in microservice architectures, introduces the Saga pattern and its execution and recovery mechanisms, compares it with two‑phase commit and TCC, and presents a centralized Saga design using ServiceComb for reliable distributed transactions.

2PCData ConsistencySaga
0 likes · 18 min read
Understanding the Saga Pattern for Distributed Data Consistency in Microservices
Code Ape Tech Column
Code Ape Tech Column
Nov 25, 2020 · Backend Development

Data Consistency Strategies in Microservices: Transaction Management and Patterns

This article reviews the evolution from traditional local and distributed transactions to BASE theory and presents four microservice data‑consistency patterns—reliable event notification, maximum‑effort notification, business compensation, and TCC—detailing their principles, advantages, drawbacks, and implementation examples.

Data ConsistencyEvent NotificationTCC
0 likes · 20 min read
Data Consistency Strategies in Microservices: Transaction Management and Patterns
DeWu Technology
DeWu Technology
Nov 23, 2020 · Backend Development

Understanding Cache: Concepts, Types, and Best Practices

Cache is a temporary storage layer that speeds data access by keeping frequently used items close to the processor, spanning hardware (CPU registers, multi‑level caches) to software (database, Redis, Memcached), and requires careful design to avoid penetration, breakdown, and consistency issues through techniques such as empty‑result caching, Bloom filters, pre‑warming, jittered expirations, logical refreshes, and multi‑level strategies.

BackendCacheData Consistency
0 likes · 12 min read
Understanding Cache: Concepts, Types, and Best Practices
Efficient Ops
Efficient Ops
Nov 4, 2020 · Fundamentals

How Journal File Systems Prevent Data Corruption After Crashes

Journal file systems use write‑ahead logging to record each write operation as a transaction, ensuring that after power loss or crashes the system can replay logs and maintain metadata and user‑data consistency, avoiding corruption and space waste through techniques like data, ordered, and metadata journaling.

Data ConsistencyWrite-Ahead Loggingfile-system
0 likes · 8 min read
How Journal File Systems Prevent Data Corruption After Crashes
Architecture Digest
Architecture Digest
Nov 3, 2020 · Backend Development

Data Consistency in Microservices: Transaction Management and Implementation Patterns

This article introduces the limitations of traditional local and distributed transactions for microservices, explains the BASE theory, and details four practical patterns—reliable event notification, maximum‑effort notification, business compensation, and TCC—providing code examples, diagrams, and a comparative table to guide developers in achieving eventual consistency across microservice architectures.

BASE theoryData ConsistencyDistributed Systems
0 likes · 19 min read
Data Consistency in Microservices: Transaction Management and Implementation Patterns
Java Architect Essentials
Java Architect Essentials
Oct 13, 2020 · Backend Development

Data Consistency in Microservices: Transaction Management Patterns and Practices

The article reviews microservice data consistency challenges, explains why traditional distributed transactions like 2PC/3PC are unsuitable, introduces the BASE theory, and details four implementation patterns—reliable event notification, maximum effort notification, business compensation, and TCC—to achieve eventual consistency.

BASE theoryData ConsistencyTCC
0 likes · 19 min read
Data Consistency in Microservices: Transaction Management Patterns and Practices
Laravel Tech Community
Laravel Tech Community
Sep 27, 2020 · Databases

Using MySQL for Persistent Data and Redis for Read‑Only Data: Consistency Strategies and High‑Concurrency Solutions

The article explains how MySQL stores persistent data while Redis serves read‑only data, outlines read/write request handling, discusses consistency issues in low and high concurrency scenarios, and provides practical solutions such as cache invalidation, queue‑based updates, and deployment considerations.

Data ConsistencyQueuecaching
0 likes · 6 min read
Using MySQL for Persistent Data and Redis for Read‑Only Data: Consistency Strategies and High‑Concurrency Solutions
Programmer DD
Programmer DD
Sep 15, 2020 · Backend Development

Solving Distributed Cache Consistency: Cache‑Aside Pattern & Lazy Update Strategies

This article explains the classic Cache‑Aside pattern, analyzes common cache‑database consistency problems in high‑traffic systems, and presents practical solutions—including delete‑first updates, internal JVM queues, lazy recomputation, and routing considerations—to ensure reliable data synchronization under heavy concurrency.

Backend ArchitectureCache AsideData Consistency
0 likes · 11 min read
Solving Distributed Cache Consistency: Cache‑Aside Pattern & Lazy Update Strategies
Big Data Technology & Architecture
Big Data Technology & Architecture
Sep 14, 2020 · Big Data

Distributed File Systems: Overview, Design Requirements, Architecture Models, and Key Considerations

This article provides a comprehensive overview of distributed file systems, covering their historical evolution, essential design requirements, centralized and decentralized architecture models, persistence, scalability, high availability, performance optimization, security, and additional practical aspects such as space allocation, file deletion, small‑file handling, and deduplication.

Data ConsistencyHigh Availabilitydistributed file system
0 likes · 21 min read
Distributed File Systems: Overview, Design Requirements, Architecture Models, and Key Considerations
Java Architect Essentials
Java Architect Essentials
Sep 1, 2020 · Backend Development

How to Ensure Data Consistency in Microservices: Patterns and Pitfalls

Microservice architectures struggle with traditional ACID transactions, so this article reviews local and distributed transaction basics, explains why 2PC/3PC are unsuitable, introduces the BASE model, and details four practical consistency patterns—reliable event, async event, business compensation, and TCC—highlighting their mechanisms, advantages, and drawbacks.

BASE theoryData ConsistencyDistributed Systems
0 likes · 17 min read
How to Ensure Data Consistency in Microservices: Patterns and Pitfalls
Full-Stack Internet Architecture
Full-Stack Internet Architecture
Aug 28, 2020 · Databases

Understanding Master/Slave Read‑Write Splitting and Data Consistency with Sharding‑JDBC

This article explains the data‑delay problem inherent in asynchronous MySQL master‑slave replication, demonstrates how Sharding‑JDBC provides read‑write splitting and forced‑master routing via HintManager, and offers configuration and code examples as well as alternative async‑query strategies for real‑time consistency.

Data ConsistencyHintManagerJava
0 likes · 8 min read
Understanding Master/Slave Read‑Write Splitting and Data Consistency with Sharding‑JDBC
dbaplus Community
dbaplus Community
Aug 27, 2020 · Backend Development

Taming Microservice Chaos: Stability, Degradation & Data Consistency

This article shares practical guidance on microservice benefits, common pitfalls such as stability and data consistency issues, and detailed solutions including circuit breakers, service degradation tactics, TCC distributed transactions, transactional messaging with RocketMQ, seamless data migration, and full‑stack APM monitoring.

APM monitoringData ConsistencyTCC
0 likes · 37 min read
Taming Microservice Chaos: Stability, Degradation & Data Consistency
58 Tech
58 Tech
Aug 21, 2020 · Backend Development

Design and Optimization Practices of the Advertising Transmission Channel at LEGO Platform

This article systematically introduces the advertising transmission process, outlines challenges of scale and stability, and presents a comprehensive set of architectural, operational, and performance optimizations—including primary‑backup high availability, stateless incremental processing, message reliability, flow tiering, indexing sharding, and schedule management—to achieve high‑throughput, low‑latency, and eventually consistent ad delivery.

AdvertisingData Consistencyscalability
0 likes · 8 min read
Design and Optimization Practices of the Advertising Transmission Channel at LEGO Platform
ITPUB
ITPUB
Aug 16, 2020 · Databases

Scalable Database Design for Likes, Comments, and Bookmarks in Mobile Apps

This article explains how to design database schemas and choose between MySQL and Redis for handling likes, comments, and bookmark features in mobile applications, covering requirements, schema examples, query patterns, scaling challenges, and data consistency considerations.

Data Consistencydatabase designlikes
0 likes · 7 min read
Scalable Database Design for Likes, Comments, and Bookmarks in Mobile Apps
Ctrip Technology
Ctrip Technology
Jul 16, 2020 · Databases

Design and Implementation of Bidirectional Redis Synchronization Across IDC: Cycle Break, LWW, Vector Clock, Tombstone, GC, and Expire

This article details Ctrip's practical design and implementation of a bidirectional Redis synchronization system across data centers, covering replication loop breaking, conflict resolution with Last Write Wins and Vector Clocks, tombstone handling, garbage collection strategies, and expiration policies to ensure data consistency.

CRDTData ConsistencyGarbage Collection
0 likes · 12 min read
Design and Implementation of Bidirectional Redis Synchronization Across IDC: Cycle Break, LWW, Vector Clock, Tombstone, GC, and Expire
Programmer DD
Programmer DD
May 21, 2020 · Backend Development

Designing a Universal Cache for Static Data in Microservices

Static business data in microservice systems, such as vehicle models and user profiles, change rarely yet demand high accuracy and real‑time access; this article proposes a universal caching architecture using a business service, persistent queue, Redis cache, and consistency checks to achieve scalable, reliable reads.

Data ConsistencyQueuecaching
0 likes · 13 min read
Designing a Universal Cache for Static Data in Microservices
Top Architect
Top Architect
May 13, 2020 · Backend Development

A General Cache Handling Mechanism for Static Business Data in Microservice Architecture

This article proposes a universal cache processing solution for low‑frequency static business data in microservice systems, detailing why caching is needed, the role of services, queues, Redis, consistency checks, and trade‑offs such as cache expiration, aiming to achieve high‑throughput, real‑time queries while ensuring data reliability.

Data ConsistencyQueue
0 likes · 13 min read
A General Cache Handling Mechanism for Static Business Data in Microservice Architecture
Architecture Digest
Architecture Digest
Apr 25, 2020 · Backend Development

Data Consistency Strategies in Microservices: Transaction Management Patterns

This article introduces the limitations of traditional local and distributed transactions for microservices, explains the BASE theory, and details four major patterns—reliable event notification, max‑effort notification, business compensation, and TCC—to achieve eventual consistency in microservice architectures.

Data Consistencydistributed transactionseventual consistency
0 likes · 16 min read
Data Consistency Strategies in Microservices: Transaction Management Patterns
JD Tech Talk
JD Tech Talk
Mar 19, 2020 · Backend Development

How Systems Empower Business Growth: Account Acquisition, Fast Business Models, and Robust System Design

The article examines how financial platforms can acquire user accounts, design fast‑perceived business experiences through account mapping, capital advances, and cross‑account interoperability, and build stable, high‑performance backend and transaction systems that balance compliance, user experience, and operational costs.

BackendData Consistencyaccounting
0 likes · 22 min read
How Systems Empower Business Growth: Account Acquisition, Fast Business Models, and Robust System Design
Architects Research Society
Architects Research Society
Mar 6, 2020 · Backend Development

Ensuring Data Consistency Across Microservices: Saga, Reconciliation, Event Sourcing, and Change Data Capture

The article explains why achieving data consistency across multiple microservices is challenging, reviews the limitations of two‑phase commit, and presents practical techniques such as the Saga pattern, reconciliation, event logs, orchestration vs. choreography, and change‑data‑capture to keep distributed systems eventually consistent.

CDCData ConsistencySaga pattern
0 likes · 12 min read
Ensuring Data Consistency Across Microservices: Saga, Reconciliation, Event Sourcing, and Change Data Capture
Alibaba Cloud Developer
Alibaba Cloud Developer
Dec 9, 2019 · Backend Development

Mastering Cache Strategies: Avoid Pitfalls and Ensure Data Consistency

This article explains core caching concepts, common pitfalls such as cache penetration, breakdown and avalanche, presents classic update patterns like Cache‑Aside, Write‑Through and Write‑Behind, analyzes consistency challenges, and offers practical guidelines for designing robust multi‑level cache architectures.

Cache AsideData ConsistencyDistributed Systems
0 likes · 29 min read
Mastering Cache Strategies: Avoid Pitfalls and Ensure Data Consistency
System Architect Go
System Architect Go
Nov 17, 2019 · Databases

Handling Single Point Failures and Disaster Recovery in InfluxDB

To mitigate the inherent single‑point‑failure risk of the open‑source InfluxDB community edition, the article proposes deploying multiple InfluxDB instances with concurrent client writes, tracking failed writes, temporarily storing them, and using custom workers to replay data, while addressing timeout, data consistency, and storage considerations.

Data ConsistencyHigh AvailabilityInfluxDB
0 likes · 3 min read
Handling Single Point Failures and Disaster Recovery in InfluxDB
Architecture Digest
Architecture Digest
Jul 2, 2019 · Fundamentals

Key Practices for High Availability, Isolation, and Data Consistency in Large‑Scale Internet Systems

The article outlines essential techniques for building highly available internet services, covering system availability metrics, multi‑level caching, database and service isolation, concurrency control, gray‑release deployment, comprehensive monitoring, graceful degradation, asynchronous design, and data‑consistency scenarios for both real‑time and offline big‑data workloads.

Data ConsistencyHigh Availabilitymonitoring
0 likes · 8 min read
Key Practices for High Availability, Isolation, and Data Consistency in Large‑Scale Internet Systems
Qunar Tech Salon
Qunar Tech Salon
May 21, 2019 · Databases

Interview with Wang Zhufeng, Qunar.com Database Director on DTCC Conference Insights and Database Evolution

In this interview, Qunar.com’s database director Wang Zhufeng shares his six‑year journey, discusses the significance of the DTCC conference, outlines the evolution of Qunar’s MySQL architecture toward clustering, cloud‑native solutions, and data consistency, and offers advice for aspiring database engineers.

DTCC ConferenceData ConsistencyDatabase Architecture
0 likes · 12 min read
Interview with Wang Zhufeng, Qunar.com Database Director on DTCC Conference Insights and Database Evolution
Architect's Tech Stack
Architect's Tech Stack
Apr 1, 2019 · Databases

Comprehensive Guide to Common Redis Issues and Their Solutions

This article provides an in‑depth overview of why Redis is used, its drawbacks, the reasons behind its single‑threaded speed, data types and use‑cases, expiration policies, memory eviction strategies, consistency challenges with databases, and practical solutions for cache penetration, cache avalanche, and concurrent key competition.

ConcurrencyData ConsistencyMemory Management
0 likes · 15 min read
Comprehensive Guide to Common Redis Issues and Their Solutions
MaGe Linux Operations
MaGe Linux Operations
Mar 27, 2019 · Backend Development

Why QQGame’s Room‑Join Failures Reveal Hidden Challenges in Scalable Backend Design

The article analyzes QQGame’s room‑entry failures caused by massive concurrent users, explores the limits of client‑side data synchronization, and proposes a scalable backend architecture using region‑based partitioning, autonomous server processing, and distributed database sharding to achieve high availability and data consistency.

Data ConsistencyLoad BalancingScalable Backend
0 likes · 14 min read
Why QQGame’s Room‑Join Failures Reveal Hidden Challenges in Scalable Backend Design
NetEase Game Operations Platform
NetEase Game Operations Platform
Mar 23, 2019 · Backend Development

Implementing a Redis‑Based Distributed Lock to Ensure Data Consistency in a CMDB System

This article analyzes data‑inconsistency problems caused by concurrent multithreaded updates in a CMDB system and presents a step‑by‑step exploration of synchronization solutions, ultimately implementing a robust Redis‑backed distributed lock in Python to guarantee atomic reads and writes while handling edge cases such as crashes and lock expiration.

CMDBData ConsistencyPython
0 likes · 17 min read
Implementing a Redis‑Based Distributed Lock to Ensure Data Consistency in a CMDB System
Tencent Cloud Developer
Tencent Cloud Developer
Mar 12, 2019 · Cloud Native

Understanding Active-Active Disaster Recovery Architecture: Challenges and Implementation Strategies

The article argues that cold backup and active‑passive setups provide false security and outlines how true active‑active disaster‑recovery requires local‑datacenter request handling, business‑driven data sharding, and low‑latency cross‑site synchronization, recommending a staged rollout from city‑level to cross‑region architectures while weighing ROI.

Data Consistencyactive-active-architecturecloud-native
0 likes · 9 min read
Understanding Active-Active Disaster Recovery Architecture: Challenges and Implementation Strategies
Xianyu Technology
Xianyu Technology
Mar 6, 2019 · Databases

Seamless Migration of Xianyu Product Database Using TDDL and DTS

Xianyu migrated its tens‑billion‑row product database from a shared MySQL cluster to an isolated instance by refactoring with dual‑write, employing TDDL for sharding and sequence management, and using Alibaba Cloud DTS for full‑load, incremental sync, consistency verification, and a no‑data‑loss rollback, achieving seamless, user‑transparent migration and improved stability.

DTSData ConsistencyDatabase Migration
0 likes · 9 min read
Seamless Migration of Xianyu Product Database Using TDDL and DTS
Efficient Ops
Efficient Ops
Mar 3, 2019 · Fundamentals

How Journal File Systems Prevent Data Loss After Crashes

Journal file systems protect against data corruption caused by power loss or crashes by recording each write operation as a transaction in a dedicated log, then committing the changes only after the log is safely stored, enabling replay to restore consistency.

Data ConsistencyWrite-Ahead Loggingfile-system
0 likes · 6 min read
How Journal File Systems Prevent Data Loss After Crashes
Youzan Coder
Youzan Coder
Jan 11, 2019 · Backend Development

Business Reconciliation Platform Architecture Design for Distributed Systems

The article describes YouZan's business reconciliation platform for distributed systems, which detects and quantifies data inconsistencies by offering easy plug‑in integration, a four‑step orchestrated workflow, high‑throughput offline processing with Spark, second‑level real‑time event handling, a three‑layer architecture, and health monitoring for transaction chains.

CAP theoremData ConsistencyDistributed Systems
0 likes · 9 min read
Business Reconciliation Platform Architecture Design for Distributed Systems
Architects' Tech Alliance
Architects' Tech Alliance
Dec 10, 2018 · Fundamentals

Why Consistency Matters in Distributed Systems: A Deep Dive

This article explains the fundamental reasons for building distributed systems, examines the inevitable side‑effects—especially data consistency challenges—analyzes the root causes of inconsistency, and walks through various consistency models from eventual to linearizability with clear examples and illustrations.

Data ConsistencyDistributed Systemsconsistency models
0 likes · 10 min read
Why Consistency Matters in Distributed Systems: A Deep Dive
Architects' Tech Alliance
Architects' Tech Alliance
Oct 12, 2018 · Operations

Understanding Microsoft Volume Shadow Service (VSS): Architecture, Components, and Backup Process

Microsoft's Volume Shadow Service (VSS) is a backup and recovery framework that creates consistent point-in-time snapshots by coordinating requestors, writers, and providers, supporting fast data backup, file-level restores, and various storage scenarios through full and copy-on-write snapshot methods.

Data ConsistencyMicrosoftStorage
0 likes · 9 min read
Understanding Microsoft Volume Shadow Service (VSS): Architecture, Components, and Backup Process
Java Backend Technology
Java Backend Technology
Apr 9, 2018 · Backend Development

How to Tackle High Concurrency: Prevent Data Chaos and Server Overload

This article explains the consequences of high‑traffic spikes, presents practical database and code‑level strategies to keep data consistent, and outlines server‑side architectures—including load balancing, caching, and Redis queues—to sustain massive concurrent requests without crashing.

Data ConsistencyLoad BalancingNode.js
0 likes · 9 min read
How to Tackle High Concurrency: Prevent Data Chaos and Server Overload
Meituan Technology Team
Meituan Technology Team
Mar 22, 2018 · Backend Development

Design and Implementation of a Reconciliation System for Meituan Delivery Settlement

The article describes Meituan’s reconciliation system for delivery settlement, which ingests diverse external data, normalizes it, performs multi‑stage offline and online verification to detect missing, duplicate, or incorrect settlements, and automatically records and resolves errors through a unified balancing center, ensuring fund safety at tens‑million‑order scale.

BackendData Consistencyfinancial safety
0 likes · 18 min read
Design and Implementation of a Reconciliation System for Meituan Delivery Settlement
Hujiang Technology
Hujiang Technology
Mar 13, 2018 · Databases

Migrating from SQL Server to MySQL: Strategies, Tools, and Lessons Learned

This article details the background, design considerations, migration workflows, tooling choices, data consistency verification, rollback mechanisms, and practical experiences of moving a large‑scale production environment from Microsoft SQL Server to MySQL, covering both offline and online migration scenarios.

Data ConsistencyDatabase MigrationETL
0 likes · 13 min read
Migrating from SQL Server to MySQL: Strategies, Tools, and Lessons Learned
JD Retail Technology
JD Retail Technology
Nov 14, 2017 · Operations

Design and Implementation of JD.com's Multi‑Active Distributed Architecture

This article details JD.com's multi-active distributed architecture, covering its evolution from single‑data‑center to multi‑region deployments, network design, leaf‑spine topology, data consistency mechanisms, application scheduling, monitoring, and disaster recovery strategies that enhance high availability and user experience.

Data ConsistencyDistributed SystemsMulti-Active
0 likes · 11 min read
Design and Implementation of JD.com's Multi‑Active Distributed Architecture
Zhuanzhuan Tech
Zhuanzhuan Tech
Oct 31, 2017 · Cloud Native

Key Technologies and Design Patterns for Implementing Microservices: Architecture Characteristics, Patterns, and Data Consistency

This article explains microservice architecture fundamentals, typical design patterns such as chain, aggregator, data‑sharing and asynchronous messaging, and presents practical solutions for service splitting, data consistency, and distributed transaction management based on real‑world experience at the ZhaiZhai platform.

Cloud NativeData ConsistencyDesign Patterns
0 likes · 9 min read
Key Technologies and Design Patterns for Implementing Microservices: Architecture Characteristics, Patterns, and Data Consistency
Architecture Digest
Architecture Digest
May 31, 2017 · Operations

Designing Distributed Transaction Architecture and Ensuring Data Consistency in a Flow Recharge System

The article explains how to break large transactions into small atomic operations combined with asynchronous messaging, describes ACID properties, presents banking and flow‑recharge scenarios, compares local and distributed (flexible) transactions, and details micro‑service architecture, compensation and async strategies to achieve eventual consistency.

Data Consistencyasynchronous notificationcompensation
0 likes · 11 min read
Designing Distributed Transaction Architecture and Ensuring Data Consistency in a Flow Recharge System
21CTO
21CTO
Apr 15, 2017 · Backend Development

How to Design Distributed Transactions for Consistent Microservices

This article explains the principles of distributed transaction design, covering ACID fundamentals, typical banking and e‑commerce scenarios, various transaction models such as two‑phase, compensation, asynchronous and best‑effort notifications, and presents a micro‑service architecture with concrete flow diagrams and scaling strategies for a high‑throughput traffic‑recharge platform.

Data Consistencydistributed transactionseventual consistency
0 likes · 12 min read
How to Design Distributed Transactions for Consistent Microservices
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Apr 11, 2017 · Databases

Why High Availability Triggers a Consistency‑Performance Trade‑off in Distributed Databases

The article explains how achieving high availability through data redundancy introduces consistency challenges that in turn affect performance, and it reviews partitioning, mirroring, consistency models, replication architectures, and two/three‑phase commit protocols in distributed systems.

Data ConsistencyDistributed SystemsHigh Availability
0 likes · 18 min read
Why High Availability Triggers a Consistency‑Performance Trade‑off in Distributed Databases
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Apr 7, 2017 · Databases

Mastering Final Consistency: Strategies for Distributed Transactions

Ensuring final data consistency in modern applications—whether enterprise or internet—requires understanding ACID, CAP, BASE theories and applying practical techniques such as single-database transactions, two-phase commit, transactional message queues, compensation tasks, async callbacks, and double-check mechanisms to build robust distributed systems.

ACIDBASECAP
0 likes · 8 min read
Mastering Final Consistency: Strategies for Distributed Transactions
Architecture Digest
Architecture Digest
Jan 21, 2017 · Backend Development

Evolution and Best Practices of the Qinglong Logistics System Architecture

The article chronicles the Qinglong logistics platform from its 2012 MVP launch through successive versions to a smart‑logistics system, detailing architectural evolution, high‑availability, performance, data‑consistency strategies, and user‑experience practices that underpin large‑scale backend development.

BackendData ConsistencyHigh Availability
0 likes · 16 min read
Evolution and Best Practices of the Qinglong Logistics System Architecture
Meituan Technology Team
Meituan Technology Team
Dec 27, 2016 · Backend Development

Ensuring Data Consistency in Meituan Hotel Direct Connection Platform

To keep its rapidly expanding hotel‑direct platform consistent despite unstable supplier interfaces, Meituan evolved from full‑batch pulls to segmented fetching, predictive trigger‑based updates, and finally supplier‑initiated pushes, creating a hybrid pull‑push architecture that ensures low‑latency, reliable product and inventory data.

Data ConsistencyMySQL replicationbackend development
0 likes · 18 min read
Ensuring Data Consistency in Meituan Hotel Direct Connection Platform
Tencent Music Tech Team
Tencent Music Tech Team
Nov 18, 2016 · Backend Development

Design and Architecture of a Live Streaming Gift System

The live‑streaming gift system is built with a producer‑consumer message queue, KV store, cache, SQL and SSD layers, employing asynchronous leaderboard updates, unique transaction IDs with redo handling, end‑to‑end encryption and replay protection to ensure high‑consistency, real‑time, secure processing of monetary gifts.

Backend ArchitectureData Consistencygift system
0 likes · 7 min read
Design and Architecture of a Live Streaming Gift System
Alibaba Cloud Developer
Alibaba Cloud Developer
Sep 5, 2016 · Operations

Avoid Common Pitfalls in Geo-Active High-Availability Design

This article examines common misconceptions in designing geo-distributed active-active systems, explains why striving for perfect real-time data sync is unrealistic, and offers practical strategies—such as prioritizing core services, reducing distance, limiting data replication, and combining storage sync with messaging—to achieve reliable high-availability.

Active-ActiveData ConsistencyHigh Availability
0 likes · 17 min read
Avoid Common Pitfalls in Geo-Active High-Availability Design
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Jun 19, 2016 · Backend Development

How to Tackle Common Cache Problems in Distributed Systems

This article explores typical cache challenges in distributed systems—including data consistency, high availability, cache avalanche, and cache penetration—explaining their causes, real‑world scenarios, and practical mitigation strategies to ensure reliable and efficient caching.

CacheData ConsistencyDistributed Systems
0 likes · 9 min read
How to Tackle Common Cache Problems in Distributed Systems
Qunar Tech Salon
Qunar Tech Salon
Apr 26, 2016 · Databases

Distributed Transaction and Data Consistency Solutions for E‑commerce Systems

The article examines the challenges of maintaining data consistency across distributed services in e‑commerce, explains strong, weak and eventual consistency models, and presents six practical solutions—including business integration, the eBay BASE pattern, Qunar, Mogujie, Alipay DTS, and Nongxin—highlighting how local transactions, idempotent messaging, and eventual consistency can replace heavyweight distributed‑transaction frameworks.

BASEData Consistencydistributed transactions
0 likes · 16 min read
Distributed Transaction and Data Consistency Solutions for E‑commerce Systems