This article explores the requirements and database design patterns for implementing likes, comments, and favorites in apps, comparing traditional MySQL schemas with Redis‑based storage and caching solutions, and discusses their trade‑offs, scalability, and data consistency considerations.
This article provides a comprehensive introduction to ZooKeeper, covering its purpose, design goals, hierarchical data model, znode types, client sessions, watch mechanism, consistency guarantees, leader election process, leader and follower workflows, and the Zab atomic broadcast protocol.
This article examines the core concepts, architectural models, scalability, persistence, high availability, performance optimization, and security considerations of distributed file systems, comparing centralized and decentralized designs such as GFS and Ceph to guide future system design decisions.
This article analyzes the core design goals, architectural models, scalability, high‑availability, consistency, security, and performance considerations of modern distributed file systems, comparing classic solutions like NFS and GFS with newer approaches such as Ceph.
This article provides a comprehensive overview of distributed file systems, covering their historical evolution, essential requirements such as POSIX compliance, persistence, scalability, and security, and compares centralized (e.g., GFS) and decentralized (e.g., Ceph) architectures, along with strategies for high availability, performance optimization, and handling small files.
This article provides a comprehensive overview of ZooKeeper, covering its design goals, hierarchical data model, session handling, watch mechanism, consistency guarantees, leader election process, and the Zab protocol that underpins its distributed coordination capabilities.
This article explains the Cache Aside pattern, why deleting stale cache entries is often better than updating them, and presents a queue‑based lazy‑update solution that handles simple and complex consistency problems in high‑concurrency environments while outlining practical performance considerations.
This article explains common Kafka interview questions by analyzing the system's high‑availability design, reliability mechanisms, consistency model, performance tricks such as sequential writes and zero‑copy, and the reasons for using Kafka and message queues, providing both conceptual insight and practical details.
This article explains the fundamentals of distributed consistency, covering weak and strong consistency, the CAP theorem, ACID and BASE models, and detailed overviews of 2PC, 3PC, Paxos, Raft, Gossip, NWR, Quorum, and Lease mechanisms, highlighting their trade‑offs and practical use cases.
This article provides a thorough overview of distributed systems, explaining their definition, when to adopt them, core concepts like partition and replication, common challenges, essential properties, typical architectural components, and practical implementations to help readers build a solid learning roadmap.
This article examines service discovery patterns, comparing client‑side and server‑side approaches, explores consistency trade‑offs, health‑check mechanisms, subscription models, graceful up/down procedures, and high‑availability designs to help engineers choose the right solution for microservice architectures.
This article explains the MySQL group_replication_consistency parameter, describes its five possible values, demonstrates the three most common consistency modes (EVENTUAL, BEFORE, AFTER) with practical multi‑node examples, and discusses their advantages, drawbacks, and impact on query latency and data integrity.
This article explains the rise of NoSQL databases, the CAP theorem and its CP/AP trade‑offs, critiques of CAP, and the BASE model, illustrating concepts with examples such as HBase and discussing how these principles guide modern distributed data systems.
This article explains how to build a MySQL‑based flash‑sale system that guarantees high performance and strong consistency, discusses common pitfalls such as overselling and duplicate purchases, and presents progressive solutions from simple unique indexes to transaction‑based locking and performance‑oriented designs that may sacrifice consistency when necessary.
This article explains how to design a high‑performance, strongly consistent flash‑sale (秒杀) system using MySQL InnoDB, covering table schema, unique indexes, transaction handling, row‑level locking, and trade‑offs between consistency and performance, with concrete SQL examples and flow diagrams.
This article explains the core concepts of replication, consistency, and fault tolerance in distributed systems, discusses strong and asynchronous replication methods, and details the CAP theorem with its consistency, availability, and partition tolerance trade‑offs, illustrating AP and CP scenarios such as Eureka and Zookeeper clusters.
This article explains the cache‑aside pattern, why deleting stale cache entries is preferred over updating them, outlines basic and complex cache‑database inconsistency scenarios, and presents a queue‑based solution with practical considerations for read‑blocking, request routing, and hotspot handling in high‑traffic systems.
This article provides a comprehensive overview of distributed file systems, covering their historical evolution, essential requirements, architectural models with and without a central node, persistence strategies, scalability, high availability, performance optimizations, security mechanisms, and practical considerations for small‑file workloads.
This article provides a comprehensive overview of distributed file systems, covering their historical evolution, essential requirements such as POSIX compliance, persistence, scalability and security, architectural models with and without a central node, replication strategies, load balancing, high availability, performance optimizations, and practical considerations for small‑file handling.
This article describes JDHBase’s large‑scale KV storage architecture, its HBase‑based asynchronous replication mechanism, multi‑active cluster design, client‑side routing via Fox Manager, automatic failover strategies, dynamic replication tuning, and serial replication techniques to ensure data consistency across geographically distributed data centers.
This article provides a comprehensive overview of distributed system fundamentals, covering node modeling, replica concepts, consistency levels, data distribution strategies, centralized and decentralized replica protocols, lease mechanisms, quorum, two‑phase commit, MVCC, Paxos, and the CAP theorem, while analyzing their trade‑offs in availability, consistency, and partition tolerance.
The article explains why Zookeeper’s CP consistency model and performance characteristics make it unsuitable for high‑availability service registries, advocating an AP‑oriented approach for service discovery in large‑scale systems.
When a user moves from an anonymous to a logged-in state, maintaining consistent feature‑flag behavior—such as discount eligibility—requires strategies like assigning a persistent visitor ID via cookies to link the two sessions and ensure a seamless experience.
This article analyzes the core challenges of building flash‑sale (秒杀) systems—high concurrency reads and writes, strict consistency, and ultra‑high availability—and presents a layered architectural approach covering dynamic/static separation, hotspot optimization, database tuning, and comprehensive high‑availability strategies.
This article provides a comprehensive overview of snapshot technology, explaining its definition, the backup challenges it addresses, the creation process, various implementations across file systems, LVM, NAS, disk arrays, virtualization, databases, and different snapshot types such as COW, ROW, incremental and continuous data protection, as well as consistency issues and solutions.
This article explains how to design a flash‑sale (秒杀) system that handles massive concurrent requests by applying dynamic‑static separation, hotspot optimization, database tuning, and traffic‑shaping techniques to achieve high performance, strong consistency, and robust high‑availability.
Redis Cluster does not guarantee strong consistency, and in scenarios like asynchronous replication or network partitions data can be lost even after client acknowledgment; using the WAIT command, configuring node timeout, and understanding master‑slave election can reduce but not fully eliminate these risks.
This article explores the architectural design of a flash‑sale (秒杀) system, covering high‑performance techniques, consistency guarantees, high‑availability strategies, hotspot optimization, and practical implementation details such as static‑dynamic separation, caching, data integration, and database tuning.
This article examines four common database architecture patterns—primary‑standby, dual‑primary, master‑slave read/write separation, and a hybrid dual‑primary + master‑slave—evaluating each against high availability, performance, consistency, and scalability, and then presents practical consistency‑resolution techniques and cache strategies.
This article explains three core design principles—consistency, fault tolerance, and readability—showing how they reduce learning costs, prevent user errors, and improve text clarity, ultimately making designs more persuasive and user‑friendly.
This article examines the architectural design of flash‑sale (seckill) systems, covering high performance through dynamic‑static separation, hotspot optimization, consistency handling for inventory deduction, and high‑availability techniques such as traffic shaping, queuing, and fallback plans.
This article explains why distributed databases need precise clocks, compares central, logical, and hybrid clock designs, and shows how hybrid logical clocks (HLC) together with two‑phase commit and other transaction techniques enable consistent, high‑throughput distributed transactions.
The article explains distributed consistency problems and the two‑phase commit (2PC) protocol using a restaurant kitchen and a multiplayer game analogy, detailing how a coordinator ensures atomic transactions across multiple services and discussing the inherent drawbacks of 2PC.
This article explains why Redis is chosen for projects—highlighting its performance and concurrency benefits, detailing its data types, expiration and eviction policies, and offering solutions to common issues such as cache consistency, avalanche, penetration, and key‑competition.
This comprehensive guide walks you through the fundamentals, design patterns, consistency models, core components, and engineering practices of modern distributed systems, helping you understand micro‑service architecture, network protocols, data management, fault tolerance, and performance optimization in cloud‑native environments.
This article explains how CB‑SQL implements follower reads by using safe (closed) timestamps, describes the CT update mechanism with a Minimum Proposal Tracker, and discusses routing, replica read validation, timestamp forwarding, range split/merge handling, and recovery strategies for consistent distributed reads.
This article examines Ctrip's approach to achieving multi‑region, bidirectional Redis replication by employing CRDT techniques and strong eventual consistency to overcome the limitations of single‑directional sync, network partitions, and data‑conflict challenges in distributed database systems.
This article distills the key recommendations from the 2017 PyCon talk “How to make a good library API”, offering a concise checklist that covers simplicity, consistency, flexibility, and security to help Python developers create intuitive, maintainable, and robust library interfaces.
This article examines common cache‑database consistency problems, explains why naive double‑write approaches fail, introduces the Cache‑Aside pattern, and proposes a queue‑based serialization solution with lazy cache updates to maintain data integrity under high‑traffic, concurrent read‑write workloads.
This article explains InnoDB's Multi-Version Concurrency Control (MVCC) architecture, detailing row formats, undo log types, ReadView visibility rules, purge processes, and related concepts such as semi-consistent reads and index condition pushdown, with code examples and diagrams illustrating each mechanism.
Snapshot technology creates point‑in‑time copies of files, directories or volumes, addressing common backup issues such as large data volumes, write‑in‑progress files, performance impact, and consistency, while offering various implementations—from file‑system and LVM to NAS, hypervisors, and databases—each with distinct advantages and trade‑offs.
This article provides a comprehensive overview of distributed‑system fundamentals, covering node concepts, failure types, replication models, consistency levels, performance and availability metrics, data‑distribution strategies, replica control protocols, lease mechanisms, quorum, two‑phase commit, MVCC, Paxos, and the CAP theorem.
This article outlines core database architecture principles—high availability, performance, consistency, and scalability—then compares four common setups (primary‑backup, dual‑master, primary‑replica, and hybrid), analyzes their trade‑offs, and presents practical consistency and cache‑sync solutions along with personal insights.
This article compares ten typical in‑memory databases, discusses their technical characteristics such as performance, ACID support and SQL compatibility, and provides comprehensive technical and non‑technical criteria as well as hardware recommendations for choosing the most suitable in‑memory database solution.
This article explains Kafka's mechanisms for ensuring data reliability through partition replicas, producer acknowledgments, and leader election, and describes how the high‑water‑mark and ISR concepts maintain strong data consistency across the cluster.
This resource compiles a complete series of articles on distributed system theory covering consistency, consensus, high availability, scalability, performance, testing, and operations, offering both quick overviews for newcomers and in‑depth readings for practitioners seeking to master modern distributed architectures.
This article explains the CAP theorem, the trade‑offs among consistency, availability and partition tolerance, describes strong, weak and eventual consistency models, and introduces the BASE approach as a practical alternative for large‑scale distributed systems.
This article explores practical cache usage by comparing DB‑first and cache‑first approaches, discusses transaction‑based safety, cache‑aside patterns, handling high‑availability databases, local‑cache decisions, and consistency solutions among local, distributed caches and databases.
This article examines core database architecture principles—high availability, performance, consistency, and scalability—and compares four common deployment patterns (primary‑standby, dual‑primary, primary‑replica with read/write separation, and a hybrid dual‑primary/replica design), followed by detailed consistency solutions and practical insights for real‑world implementation.
This article outlines core database architecture principles—high availability, performance, consistency, and scalability—examines four typical deployment schemes with their trade‑offs, and presents multiple consistency‑preserving strategies for both primary/replica and DB‑cache interactions.
This article provides a comprehensive analysis of mainstream microservice service‑registry products—including Nacos, Eureka, ZooKeeper, and Consul—covering their data models, consistency protocols, load‑balancing strategies, health‑check mechanisms, performance, scalability, usability, and extensibility to guide practitioners in selecting and designing registration centers.
This article explains fundamental database architecture principles, compares four common deployment patterns—including primary‑backup, dual‑primary, master‑slave with read‑write separation, and a hybrid dual‑primary/master‑slave design—analyzes their high‑availability, performance, consistency, and scalability characteristics, and presents practical consistency‑resolution techniques and personal insights.
This article provides a thorough introduction to distributed systems, covering fundamental theories such as nodes, networks, time, ordering, consistency models (CAP, ACID, BASE), design patterns, scenario classifications, engineering practices, and the full technology stack needed to build and operate microservice‑based distributed applications.
This article introduces the foundational concepts and principles of distributed systems—including basic concepts, consistency models, CAP theorem, logical clocks, and advanced protocols like Paxos, Raft, and Zab—serving as the first part of a comprehensive Big Data mastery series.
The article explains how the CAP theorem forces trade‑offs between consistency, availability and partition tolerance, then surveys distributed commit protocols (2PC, 3PC) and consensus algorithms (Paxos, Raft, Zookeeper’s ZAB), and shows their practical use in systems such as ZanKV that combine Raft with RocksDB for strongly consistent, fault‑tolerant key‑value storage.
The article explores the fundamental problems of UI engineering—consistency, responsiveness, latency, navigation, caching, accessibility, internationalisation, transport, resilience, abstraction, and prioritisation—encouraging developers to solve them by building small, library‑free prototypes to deepen their understanding of trade‑offs.
This article examines Qunar.com’s internally developed message queue (QMQ), discussing the motivations behind building it, the challenges of consistency and storage, client‑side transaction integration, the novel log‑based storage model, and its support for arbitrary delayed messages.
This article distills key recommendations from a 2017 PyCon talk into a practical checklist for building Python library APIs, covering simplicity, consistency, flexibility, and security to help developers create intuitive, reliable, and well‑structured interfaces.
The article explains distributed final consistency challenges, compares heavyweight transaction frameworks with lightweight techniques such as idempotency, retries, state machines, recovery logs, and async verification, and outlines CAP, BASE principles and practical implementation steps for backend systems.
This article explains the CAP theorem, its implications for distributed systems, details the Paxos consensus algorithm and its availability optimizations, and introduces ZooKeeper's ZAB protocol, showing how they together address consistency‑availability trade‑offs in real‑world deployments.
The article explains major cache issues such as consistency, concurrency, penetration, jitter, avalanche, and the “bottomless pit” phenomenon, and outlines practical techniques like locking, empty‑object caching, request filtering, consistent hashing, rate limiting, and multi‑level caching to prevent system failures.
This article explains common cache problems such as consistency, concurrency, penetration, jitter, avalanche, and the bottomless‑hole phenomenon, and provides practical strategies like expiration policies, locking, empty‑object caching, sharding, rate limiting, and IO optimizations to keep high‑traffic systems stable.
This article explains the major cache challenges—including consistency, concurrency, penetration, jitter, avalanche, and the so‑called bottomless‑pit phenomenon—describes why they occur in high‑traffic systems, and offers practical mitigation techniques such as locking, empty‑object caching, request filtering, sharding, and multi‑level caching.
This article provides a comprehensive overview of distributed file systems, covering their historical evolution, essential requirements, architectural models (centralized and decentralized), persistence strategies, scalability, high availability, performance optimization, security mechanisms, and additional considerations such as space allocation, file deletion, small‑file handling, and fingerprint‑based deduplication.
This article explains 58.com’s database architecture, covering availability through replication and dual‑master setups, read‑performance enhancements with indexing, read replicas and caching, consistency solutions, rapid horizontal scaling methods, and a review of Codd’s twelve rules for relational design.
The article explains various cache problems—including consistency, concurrency, penetration, avalanche, and the bottomless‑pit phenomenon—and presents practical mitigation techniques such as active updates, locking, empty‑object caching, request filtering, consistent hashing, and multi‑level caching to ensure reliable high‑performance systems.
This article analyzes the principles, advantages, and drawbacks of distributed locking with Redis and Zookeeper, comparing single‑node and cluster scenarios, discussing RedLock, EPHEMERAL znodes, and practical failure cases to help engineers choose the right solution.
This article provides a comprehensive overview of Redis, explaining why it’s chosen for performance and concurrency, how its single‑threaded design achieves speed, the various data types and their use cases, expiration and eviction strategies, consistency challenges with databases, and solutions for cache penetration, avalanche, and key‑concurrency issues.
This article examines three common cache‑update strategies, explains why some lead to data inconsistency, and proposes practical solutions such as delayed double‑delete, asynchronous retries, and message‑queue based recovery to maintain cache‑database consistency in high‑concurrency systems.
This article provides a comprehensive overview of Redis, covering its performance benefits, single‑threaded speed, data types, expiration policies, memory eviction strategies, consistency challenges with databases, and practical techniques for handling cache penetration, snowball effects, and concurrent key conflicts.
The article explains the CAP theorem, its three properties—consistency, availability, and partition tolerance—how they influence database design, compares CA, CP, and AP classifications with examples, and discusses related concepts such as ACID, BASE, and practical design choices for distributed systems.
This article examines three common cache‑update approaches—updating the cache after the database, deleting the cache before the database write, and deleting after the database write—analyzes their drawbacks, and presents improved solutions such as delayed double‑delete, asynchronous retries, and binlog‑driven mechanisms.
This article examines three common cache‑update strategies, explains their concurrency and performance drawbacks, and proposes practical improvement techniques such as delayed double‑delete, asynchronous retries, and binlog‑based messaging to maintain consistency between cache and database.
This article provides a comprehensive review of Redis, covering why it’s used, its performance advantages, single‑threaded speed, data types and use cases, expiration policies, memory eviction strategies, consistency challenges, and practical solutions for cache penetration, avalanche, and concurrent key competition.
This article examines three common cache‑update approaches—updating the cache after the database, deleting the cache before updating the database, and updating the database then deleting the cache—analyzes their drawbacks, and presents practical solutions such as delayed double‑delete and retry mechanisms to ensure data consistency.
The article explains how distributed transactions work in microservice architectures, using inventory‑order examples and a World of Warcraft raid analogy to illustrate the XA two‑phase commit protocol, its normal and failure flows, limitations, and alternative approaches such as three‑phase commit, message‑queue and TCC transactions.
This article, based on a 2017 PyCon talk, presents a comprehensive checklist for building Python library APIs, covering simplicity, consistency, flexibility, and security with concrete examples and actionable guidelines to help developers create intuitive and reliable interfaces.
This article explains why monolithic applications need to be split, outlines the preparatory steps for understanding business complexity and defining service boundaries, and provides detailed practical guidance on database vertical and horizontal splitting, global ID generation, migration, cut‑over strategies, consistency, and post‑split stability.
This article explores the importance of data consistency in distributed systems, illustrating real‑world scenarios, explaining consistency models such as strong, weak and eventual, and detailing the challenges and theories like CAP and BASE that guide system designers in balancing consistency, availability, and partition tolerance.
This article explains the core challenges of achieving consistency in distributed systems, describes consensus algorithms such as Paxos and Raft, discusses theoretical limits like the FLP impossibility and CAP theorem, and shows how trade‑offs among consistency, availability, and partition tolerance shape practical system design.
Database sharding, or splitting databases into multiple instances and tables, is essential for scaling backend systems; this article explains its importance, historical context, core concepts, storage stretching techniques, index and consistency handling, and provides a practical design example for transaction systems.
This article explains the motivations behind database sharding, defines the concept, explores its historical background, discusses storage stretching, indexing and consistency challenges, and presents a practical sharding design example for transaction systems.
This article explains why Xiaomi built Pegasus to replace HBase, describes its architecture—including MetaServer, ReplicaServer, partitioning, multi‑replica design and the PacificA consensus algorithm—covers implementation challenges such as load balancing, consistency, latency, testing, and outlines current status and future plans.
This article examines various consistency solutions for distributed systems, including strong and eventual consistency, local database transactions, two‑phase commit, TCC, rollback mechanisms, local message tables, and compensation techniques, illustrating their trade‑offs and appropriate application scenarios.
This article explains the challenges of consistency in distributed systems, introduces the CAP theorem and ACID properties, describes common distributed transaction techniques such as local message tables, transactional message middleware like RocketMQ, and details the two‑phase and three‑phase commit protocols with their advantages and drawbacks.
This article explains the core concepts of distributed system consistency—including the CAP theorem, ACID properties, various distributed transaction techniques, and the two‑phase and three‑phase commit protocols—while illustrating practical implementations with message queues and local message tables.
This article explains the concept of linearizability, contrasts it with serializability, and describes how Google Spanner uses the TrueTime API and commit‑wait mechanisms to provide external consistency and reliable snapshot reads in a globally distributed database system.
PacificA is Microsoft’s generic replication framework for large‑scale distributed storage systems that provides strong consistency, separates configuration management from data replication, and uses a primary‑secondary model with lease‑based fault detection to ensure availability, correctness, and efficient operation across heterogeneous nodes.
This article examines the challenges of consistency in large‑scale distributed service systems, presents real‑world case studies such as payment transfers and order processing, and outlines practical patterns—including ACID/BASE theory, two‑phase and three‑phase commit, TCC, query, compensation, periodic reconciliation, and reliable messaging—to help engineers design robust, eventually consistent architectures.
This article examines the challenges of achieving consistency in large‑scale distributed service systems, outlines common inconsistency scenarios such as split‑brain and lost updates, and presents practical patterns—including ACID/BASE trade‑offs, two‑phase and three‑phase commit, TCC, query, compensation, and reliable messaging—to guide engineers in designing robust, eventually consistent architectures.
An in‑depth review of the heated debate between Redis creator antirez and distributed‑systems expert Martin Kleppmann over the safety of Redis’s Redlock algorithm, covering single‑node lock pitfalls, failover issues, timing assumptions, fencing tokens, and practical recommendations for when to use Redlock versus simpler locks.
The article explains that Paxos relies on a fixed set of voters (the majority) for consistency, shows why configuration‑driven member changes break the protocol, and introduces a simple dynamic membership algorithm that uses delayed activation windows to achieve atomic member updates without violating Paxos guarantees.
This article details how WeChat transformed its massive memory‑cloud storage by replacing the QuorumKV NWR protocol with a non‑lease Paxos design, optimizing PaxosLog, adopting DirectIO on HDDs, and implementing operational safeguards, resulting in dramatically lower latency, higher availability, and reduced failure rates.
This article explains how PhxSQL prioritizes strong linearizable consistency, high availability, serializable isolation, and full MySQL compatibility, and why it deliberately forgoes features such as multi‑write, sharding, and strict serializable isolation due to the high cost of distributed transactions and protocol complexity.
This article traces Facebook's evolution from a small social site to a global platform, explains how its massive data‑storage challenges led to the adoption of NoSQL solutions like Cassandra and HBase, and breaks down the core patterns, consistency models, and scaling techniques that power such large‑scale systems.
This article explains the CAP theorem—its three core properties, why a distributed system can satisfy at most two of them, the formal proof, practical strategies for handling partitions, and how modern databases and architectures attempt to work around its constraints.
This article shares Qunar's architecture and practical experience in designing a self‑developed high‑availability message middleware, covering its role in transaction processing, consistency guarantees, fault‑tolerance mechanisms, isolation, monitoring, and consumer design, and discusses trade‑offs and operational considerations.
Designing Quanmin K‑Song’s high‑scale messaging system requires careful capacity estimation of throughput, storage and network traffic, robust consistency via unique transaction IDs and operation logs, and avalanche prevention through selective retries, scaling and priority‑based throttling to maintain reliability under load.
This article explains why caching is essential for high‑performance data retrieval, compares LRU and LFU eviction policies, presents three Redis‑based cache implementations, and discusses consistency challenges and solutions such as eviction ordering, consistent hashing, and delayed eviction in distributed systems.
This article compares NoSQL and SQL databases, highlighting NoSQL’s strengths in scalability, flexible schemas, and performance, while also discussing its lack of unified standards, operational complexity, and why SQL remains essential for strong consistency and transactional integrity in critical systems.
