This article explores how caching accelerates read/write performance and reduces backend load, analyzes its benefits and costs, and presents practical strategies for cache updates, granularity control, penetration, avalanche, and hot‑key issues with concrete examples and diagrams.
This article explains how data consistency is achieved through local ACID transactions, global XA/2PC mechanisms, CAP and BASE theories, and practical patterns such as reliable events, TCC, and SAGA, providing a comprehensive overview for architects designing distributed systems.
The iQIYI Basic Data Platform unifies internal data exchange standards, integrates massive multi‑business data, and implements high‑availability solutions for ID services, messaging, HBase storage, and read‑write scaling, showcasing practical engineering approaches to big‑data reliability and performance.
This article explains how caching improves application performance and reduces backend load, and it details practical optimization methods such as benefit‑cost analysis, update policies, granularity control, penetration protection, bottomless‑hole mitigation, avalanche prevention, and hot‑key rebuild strategies, primarily using Redis.
This article explains the architecture and key features of NetEase Cloud's local server‑side recording cluster, detailing how dynamic scaling, multi‑backup high availability, load‑balancing strategies, monitoring, and an embedded registration center enable secure, reliable, and scalable recording for data‑sensitive applications.
This article explains the concept of anti‑entropy as a key mechanism for achieving eventual consistency in distributed database systems, illustrates how XDB Enterprise handles node failures and data drift, and shows practical examples of repairing inconsistencies using the AE service.
This article explains the concept of eventual consistency in distributed databases, introduces the Hint Switch (HH) queue used by XDB Enterprise to handle failed writes, and discusses how replication factor and node failures affect data consistency, illustrated with practical examples and diagrams.
This article explains the fundamentals of load balancing, covering why it is needed for high‑traffic services, the difference between vertical and horizontal scaling, various load‑balancing techniques (DNS, HTTP, IP, link‑layer, hybrid), common algorithms, and the trade‑offs of software versus hardware solutions.
Designing a distributed system involves overcoming major challenges such as heterogeneity, transparency, openness, concurrency, security, scalability, and fault tolerance, each of which must be addressed to build a reliable, extensible, and performant system.
This article provides a comprehensive overview of Kafka, covering its layered architecture with Zookeeper, core concepts such as topics, partitions and consumer groups, communication workflow, partition selection strategies, rebalancing mechanisms, reliability configurations, replica synchronization, and reasons for moving away from Zookeeper, all explained in clear English.
This article explains how to design a high‑concurrency system by selecting appropriate server architecture, load balancing, database clustering, caching layers, message‑queue handling, static‑content delivery, service‑oriented decomposition, redundancy, automation, and monitoring to ensure smooth operation under heavy user traffic.
This article provides a comprehensive guide to RocketMQ, covering Windows installation, environment configuration, startup commands, visual monitoring integration, SpringBoot dependency setup, core components, message models, reliability strategies, idempotent processing, transactional messaging, and ordering guarantees for distributed systems.
This article explains how to design and evolve server architectures, load‑balancing, database clustering, caching, message queues, and other techniques to handle high‑concurrency scenarios such as flash sales and timed red‑packet distribution in large‑scale e‑commerce systems.
This article explains the concepts of sequential, causal, and eventual consistency models, illustrating their definitions, guarantees, and practical examples—including process interactions and real‑world scenarios—while comparing their strengths, trade‑offs, and related consistency guarantees such as monotonic reads, writes, and read‑your‑writes.
This article explains how caching can accelerate read/write performance and reduce backend load, analyzes its benefits and costs, discusses update strategies, granularity control, and solutions for common problems such as cache penetration, empty‑hole, avalanche, and hot‑key reconstruction, providing practical guidelines for robust backend design.
This article explains how to design and optimize backend server architecture for high‑concurrency scenarios, covering load balancing, database master‑slave clusters, NoSQL caching, concurrency testing tools, caching strategies, message‑queue based async processing, layered and distributed designs, redundancy, automation, and service‑oriented approaches.
This article explains why rate limiting is essential for system stability, outlines common throttling patterns such as circuit breaking, degradation, delayed processing, and privilege handling, and dives into popular algorithms like counter, leaky bucket, and token bucket with concrete Java and Nginx examples.
The article explains consistency concepts in distributed systems—including strong and weak (eventual) consistency, CAP and FLP theorems, and key protocols such as 2PC, 3PC, and Paxos—detailing their mechanisms, advantages, and challenges.
The article explains how to design a high‑concurrency system by planning server architecture, load balancing, master‑slave databases, NoSQL clusters, caching layers, message‑queue based asynchronous processing, static‑content CDN, and automated redundancy to ensure scalability and reliability.
This article reviews the 13‑year evolution of Ant Group's registration center, analyzes the scaling and reliability challenges of multi‑cluster service discovery, and explains how the SOFARegistry 6.0 redesign—featuring meta‑driven consistency, slot‑based sharding, application‑level discovery, chaos testing, and automated operations—addresses those challenges while preparing the project for open‑source community growth.
This article introduces the fundamentals of message queues and compares five popular products—ActiveMQ, RabbitMQ, ZeroMQ, Kafka, and RocketMQ—detailing their features, typical use cases, and providing curated learning resources for each.
The article explains the characteristics of flash‑sale scenarios, presents core design principles such as rate limiting, peak shaving, asynchronous processing and scalability, and details a complete backend and frontend architecture—including Redis‑based queueing and caching—to build a robust, high‑throughput seckill system.
This article provides a comprehensive technical summary of large‑scale distributed website architecture, covering characteristics, goals, architectural patterns, performance, high‑availability, scalability, extensibility, security, agility, and a detailed evolution roadmap with practical examples and recommendations.
This article provides a comprehensive analysis of ZooKeeper 3.5.5 source code, covering the distributed‑mode server startup sequence, leader election algorithm, cluster data synchronization, server role responsibilities, and the request‑processor pipeline, offering practical insights for developers and operators.
This article explains how to evolve a large‑scale website architecture through stages such as initial single‑server setups, application‑data separation, caching, server clustering, read‑write separation, CDN/reverse proxy, distributed storage, micro‑services, and automation to achieve high availability, scalability, performance and security.
This article compares Apache Doris and ClickHouse across architecture, deployment, multi‑tenant management, data import, storage, query capabilities, performance testing, and cost, providing practical guidance for selecting the most suitable analytical database in large‑scale OLAP scenarios.
The article explains the functional purpose, early evolution, layered and clustered design, distributed technical architecture, caching, asynchronous checks, heterogeneous storage, payment solutions, and anti‑bot measures of a cloud‑based shopping cart, highlighting stability, performance, elasticity, and fault‑tolerance.
This article analyzes the evolution of large‑scale website architecture, covering stages from single‑server setups to layered, distributed, and clustered designs, and explains how caching, read‑write separation, CDN, asynchronous messaging, redundancy, automation, and security collectively achieve high performance, availability, scalability, and extensibility.
This article presents a comprehensive guide to designing high‑concurrency backend systems, covering server architecture, load balancing, database and NoSQL clustering, caching strategies, concurrency testing tools, message‑queue solutions, first‑level cache, static data handling, layering, distribution, asynchronous processing, redundancy and automation.
This article analyzes the extreme‑traffic problem of Chinese train ticket sales, presents a multi‑layer load‑balancing architecture with Nginx weighted round‑robin, demonstrates Go and Redis code for local and global stock deduction, and shows benchmark results proving that a single machine can handle over 4 000 requests per second while preventing oversell and few‑sell.
This comprehensive guide explains the essence of software architecture, its layers and classifications, architectural levels, strategic versus tactical design, evolution from monoliths to micro‑services, common pitfalls, and key metrics for evaluating a sound architecture.
The article explains edge computing as a new, global computing model that extends cloud computing by bringing compute, storage, and services closer to users, detailing its differences from centralized computing, its resource‑edge and global‑resource characteristics, and several reference architectures such as ETSI, Intel MEC, ECC, and OpenFog.
This article examines Martin Kleppmann’s view of Kafka as a database, breaking down how Kafka handles atomicity, consistency, isolation, and durability through event streams and partitioning, and argues why MySQL still remains essential for many workloads.
This article provides a thorough overview of distributed systems, explaining their definition, core concepts such as partition and replication, key challenges, essential characteristics, typical components and protocols, a practical request flow example, and a curated list of real‑world implementations to help readers build a solid learning roadmap.
This article provides a comprehensive overview of the design principles, architectural patterns, performance, availability, scalability, security, and operational considerations for building large distributed web sites, illustrated with a step‑by‑step evolution from a single‑server setup to a multi‑layer, cloud‑native architecture.
This article explains Kafka’s core architecture—including topics, partitions, replication, consumer groups, controller coordination with Zookeeper, and performance tricks like sequential writes and zero‑copy—to show how it achieves high‑throughput, fault‑tolerant messaging for large‑scale systems.
This article presents a code-less business process orchestration framework that transforms static Go code into configurable drag‑and‑drop workflows, detailing the underlying concepts, architecture, core Go data structures, dynamic configuration, and practical deployment examples for distributed task scheduling and execution.
This article explains the core concepts of Elasticsearch’s distributed architecture—including nodes, indices, shards, replicas—and compares mixed and tiered deployment models, while also discussing data storage strategies, replica benefits, and the trade‑offs of local‑file versus shared‑storage distributed systems.
This article explains the concepts of high concurrency, high performance, and high availability, and demonstrates how to design cloud‑native IaaS, PaaS, and SaaS layers to keep internet services resilient during massive traffic spikes such as Double‑11.
This article explores service governance fundamentals, examines why etcd’s Raft‑based consensus and BoltDB storage make it ideal for large‑scale systems, compares it with ZooKeeper and Consul, and shares Baidu’s practical architecture, performance tricks, and operational metrics for high‑availability, high‑performance service management.
The article explains the reasons for scaling, outlines hardware and component expansion strategies, introduces the AKF splitting principle for distributed systems, discusses database clustering and distributed ID generation methods such as UUID and Snowflake, and describes elastic scaling challenges and solutions.
This article explains why scaling is necessary, outlines hardware and component expansion strategies, introduces the AKF partitioning principle for horizontal and vertical scaling, discusses challenges after splitting, and reviews database clustering and distributed ID generation techniques such as UUID and Snowflake, highlighting their advantages and drawbacks.
This article explains how JD.com designs and evolves its OLAP platform, covering data sources, ingestion, storage, real‑time and offline processing, key challenges such as timeliness, high throughput, consistency, and the solutions implemented to support massive e‑commerce analytics.
The article introduces Alibaba's Elastic Federated Learning Solution (EFLS), describing its business motivations, core functionalities, system architecture, sample‑set intersection, federated training pipeline, novel algorithms, product console, and future roadmap for privacy‑preserving advertising in large‑scale sparse scenarios.
This article presents a comprehensive guide to designing high‑concurrency systems, covering server architecture, load balancing, database clustering, caching strategies, message‑queue based async processing, static data offloading, service‑oriented design, redundancy, automation, and practical implementation examples for large‑scale web applications.
This article analyzes the challenges of multi‑user WebSocket communication in a clustered environment, compares Netty and Spring WebSocket implementations, and presents two practical solutions—session broadcast and a consistent‑hashing based routing scheme—complete with code samples, gateway configuration, and load‑balancing considerations.
This article explains Kafka's evolution—highlighting the introduction of Kafka Streams, the shift to a full distributed stream processing platform, practical learning paths, source‑code reading tips, common pitfalls, and the major new features introduced in Kafka 3.0.
This article explores practical approaches for enabling multi‑user communication in a distributed WebSocket cluster, covering session handling challenges, architecture design, technology stack choices, Netty versus Spring WebSocket implementations, consistent‑hashing load balancing, and detailed code examples for Spring Cloud Gateway and Ribbon integration.
The article explains the importance of isolating resources such as CPU, network, and disk in distributed architectures, describes thread, process, cluster, data‑read/write, static, and crawler isolation methods, and provides concrete code examples and best‑practice recommendations for backend developers.
This article reviews the evolution of RocketMQ from its early MetaQ roots through the 4.x releases, explains the motivations behind RocketMQ 5.0, and details its cloud‑native architecture, lightweight SDK, storage‑compute separation, POP consumption model, elastic scaling, and the upcoming RocketMQ Streams framework.
This article explains the principles behind geo‑active‑active (multi‑active) architectures, covering system availability metrics, redundancy strategies from single‑node backups to same‑city and cross‑city active‑active deployments, data‑sync challenges, routing and sharding techniques, and how these designs dramatically improve reliability and scalability.
The article reviews the most frequent errors encountered while adopting microservice architecture—excessive customization, uncontrolled complexity, and vague definitions—illustrating each with real‑world experiences, a saga implementation using Redis streams and Go, and practical advice to avoid similar pitfalls.
This article explains the fundamentals of Jepsen testing, compares it with TLA+, describes its architecture and workflow, illustrates how to apply Jepsen for linearizability verification of distributed systems such as locks, and offers practical guidance on integrating Jepsen or building custom testing frameworks.
The article explains how Ant Financial scales Double‑11 payment traffic to hundreds of thousands of TPS by employing logical data centers (LDC), unit‑based system design (RZone, GZone, CZone), database sharding, CAP theorem analysis, Paxos‑based consensus, and the OceanBase distributed database, while also detailing disaster‑recovery and traffic‑shifting mechanisms.
This article explains why rate limiting is essential for system stability, compares circuit breaking, service degradation, delayed processing, and privileged handling, details counter, leaky‑bucket, and token‑bucket algorithms, and provides concrete Java, Guava, and Nginx‑Lua code examples for practical deployment.
JD's OLAP platform runs on ClickHouse and Doris across 3,000 servers, handling billions of daily queries and petabytes of data, and this article details the selection criteria, cluster deployment models, high‑availability architecture, operational challenges, and future roadmap.
This article introduces core concepts of distributed systems, including node and replica models, various consistency levels, data distribution strategies, lease and quorum mechanisms, replica control protocols such as primary‑secondary, two‑phase commit, MVCC, Paxos, and the CAP theorem, providing a comprehensive overview for architects.
Learn how to build a pure Go instant messaging platform with a three‑layer, distributed architecture—comet, logic, and job—featuring RPC communication, Redis‑based message pushing, and cluster support, plus step‑by‑step deployment instructions, required packages, and future enhancement plans.
The article examines Kafka’s high‑availability mechanisms, detailing its multi‑replica design, ISR synchronization, leader election, and the critical role of the __consumer_offset topic, and explains why a single broker outage can render the entire cluster unusable unless replication factors are properly configured.
This article explains Kafka's internal topics, preferred replicas, partition assignment processes, log directory layout, index files, offset and timestamp lookup, log retention and compaction policies, storage architecture, delayed operations, controller role, legacy consumer design flaws, rebalance workflow, and producer idempotence, providing a comprehensive overview of Kafka's backend architecture.
This article presents a comprehensive case study of the technical challenges and optimization strategies—including traffic surges, cache penetration, cache breakdown, limit‑purchase handling, and inventory deduction—encountered during a large‑scale ticket reservation event, and demonstrates how systematic backend improvements achieved over 50% performance gains and 98% cache hit rate.
This article explains how Elasticsearch implements near real-time search by using immutable inverted indexes, segment merging, sharding, and a translog for durability, detailing the challenges and solutions behind its distributed full‑text search architecture.
Message middleware guarantees at-least-once delivery, but duplicate deliveries can cause issues; this article explains RocketMQ’s three duplication scenarios, explores simple and advanced deduplication strategies—including database-transaction and non-transactional approaches using Redis—and provides practical code samples for implementing reliable exactly-once processing.
This article provides a comprehensive overview of large‑scale distributed website architecture, covering its characteristics, design goals, architectural patterns, performance, high‑availability, scalability, extensibility, security, agility, evolution stages, and practical implementation techniques such as caching, load balancing, database sharding, service‑orientation and message queues.
This article examines why failure is inevitable in software systems, proposes a mindset of failure‑oriented design, outlines organizational roles and processes to mitigate incidents, and presents concrete technical techniques such as distributed locking and traffic shaping to build resilient, high‑availability services.
This article examines the challenges of Raft’s two‑stage Joint Consensus member changes, explores single‑step alternatives, analyzes ZooKeeper’s approach, and proposes improvements that combine safety with reduced log overhead, offering practical insights for distributed system engineers seeking more efficient reconfiguration methods.
The article outlines the characteristics of high‑traffic websites and traces the architectural evolution from a single‑server model to distributed, cache‑enabled, load‑balanced, and service‑oriented designs, highlighting techniques such as clustering, read/write splitting, CDN, NoSQL, and business micro‑services.
The Baidu Search Closed‑Door Technical Symposium, the first core technical forum hosted by Baidu’s Search Architecture Department, brings senior engineers and junior backend developers together to discuss semantic retrieval, data‑driven big‑data processing, and vertical search offline architecture, while offering limited‑capacity sessions, networking gifts, and travel subsidies.
This article explores how ElasticSearch delivers near real‑time search by leveraging immutable inverted indexes, segment merging, shard distribution, and a write‑ahead translog, detailing the challenges of persistence, disk I/O, and data loss prevention in a distributed environment.
This article describes Baidu's design and implementation of a cloud‑native, high‑performance KV storage platform—UNDB—covering the performance and cloud‑native challenges, engine optimizations, dynamic management, multi‑model architecture, and the resulting cost and reliability improvements for massive search and feed workloads.
This article provides a comprehensive comparison between RocketMQ and Kafka, covering data reliability, performance, queue capacity, real‑time delivery, retry mechanisms, ordering guarantees, scheduled messages, transactional support, query capabilities, message tracing, consumer parallelism, filtering, and commercial backing, helping engineers choose the right messaging middleware for their workloads.
The article examines how Postal Savings Bank of China tackled legacy core banking limitations by designing a distributed, cloud‑native core system using openGauss, Kunpeng hardware, microservices, and DevOps, detailing the architecture, deployment process, performance gains, and industry implications.
This article provides a comprehensive technical analysis of Kafka’s Controller component, covering its background, core responsibilities, data storage, election process, version‑specific improvements, monitoring techniques, and key source‑code excerpts to help engineers understand and manage Kafka clusters effectively.
This article walks through the challenges of sharing HTTP session data between micro‑services in a distributed SpringBoot architecture, demonstrates a Redis‑based manual solution, introduces Spring Session for transparent session clustering, and resolves OpenFeign cookie loss with a request interceptor.
This article explains the essence of software architecture, defining systems, subsystems, modules, components, frameworks, and various architectural layers such as business, application, data, code, technical, and deployment, and discusses architectural evolution from monolithic to distributed and microservice designs, including common pitfalls and recommended reading.
This article introduces the concepts, core features, deployment environments, architectures, high‑availability solutions, advantages and disadvantages of four widely used open‑source distributed message queue systems—Kafka, ActiveMQ, RabbitMQ, and RocketMQ—and provides a comparative analysis to help practitioners choose the right middleware for their distributed applications.
This article examines the challenges of mutual exclusion and idempotency in distributed environments, explains their underlying principles, compares multithread and multiprocess approaches, and presents practical distributed‑lock implementations such as Zookeeper, Redis, Tair, and the Cerberus framework, as well as the GTIS solution for idempotency.
The article compares the work attitude, technical depth, product awareness, and leadership responsibilities of senior developers and software architects, offering practical advice and concrete steps for engineers who want to upgrade their skills and transition into an architectural role.
This article provides a comprehensive interview‑oriented overview of message queues, covering their use cases, popular frameworks, selection criteria, message models, reliability techniques, ordering, throughput optimization, high availability, and common Kafka interview questions.
This article explains the common misconceptions about software architects, outlines the essential technical and soft‑skill requirements for senior developers to become architects, describes practical upgrade paths, and details the typical responsibilities and daily activities of an architect.
This article explains why rate limiting is needed in both physical venues and online systems, describes common limiting strategies such as circuit breaking, service degradation, delayed processing, and privileged handling, and details three major algorithms—counter, leaky bucket, and token bucket—along with practical Java and Nginx‑Lua code examples.
This article provides a comprehensive overview of distributed system fundamentals, covering system models, node concepts, failure types, replication strategies, consistency levels, data distribution methods, replica control protocols such as primary‑secondary, two‑phase commit, lease‑based caching, quorum, MVCC, Paxos, and the CAP theorem.
This article outlines the progression of software architecture from traditional monolithic systems through distributed applications and microservices to modern serverless designs, highlighting each model's structure, benefits, drawbacks, and typical use cases for developers and architects.
This article surveys the historical development of command‑and‑control software for unmanned aerial vehicles, compares early, first‑ and second‑generation drone architectures, and examines modern open‑control platforms, distributed UAV frameworks, behavior‑based controllers, and future trends in UAV software design.
This talk shares practical high‑availability strategies learned from years of building Taobao’s massive e‑commerce platform and migrating to Alibaba Cloud, covering traditional IDC stability mechanisms, cache and disaster‑recovery designs, cloud‑native fault‑tolerance, capacity planning, rate‑limiting, graceful degradation, and multi‑region resilience.
The article explains why resource isolation is essential for distributed architectures, describes common isolation methods such as thread, process, cluster, data, static and crawler isolation, and provides concrete examples and code snippets for Netty, Dubbo, Tomcat, and Hystrix to illustrate practical implementations.
The article explains why session sharing is critical in micro‑service and distributed deployments and presents four backend solutions—Nginx ip_hash load balancing, Tomcat session replication, Redis‑based centralized session storage, and cookie‑based sessions—detailing their implementations, advantages, and drawbacks.
Airbnb replaced duplicated, latency‑prone authorization checks in its new service‑oriented architecture by moving them into data services and building Himeji, a Zanzibar‑inspired centralized permission store that uses triple‑based policies, configurable unions, sharded caching, and Aurora backing to deliver sub‑10 ms latency for millions of checks per second with 99.999 % availability.
This article explains why session sharing is required in micro‑service and distributed environments and presents four practical solutions—Nginx ip_hash load balancing, Tomcat session replication, Redis centralized session cache, and cookie‑based sharing—detailing their implementations, code snippets, and pros and cons.
This article provides a comprehensive overview of Dubbo, the high‑performance Java RPC framework, covering its core components, advantages over alternatives, supported protocols, configuration methods, startup behavior, integration with Spring Boot, and detailed architecture for building scalable distributed services.
This article explores sixteen practical Redis use cases—including caching, distributed sessions, locks, global IDs, counters, rate limiting, bitmaps, shopping carts, timelines, message queues, lotteries, likes, tags, product filtering, follow relationships, and ranking—detailing data types, commands, and code snippets for each scenario.
This article explains why message middleware guarantees at‑least‑once delivery, the problems caused by duplicate consumption, and presents both transaction‑based and non‑transactional deduplication solutions—including a Redis‑backed idempotent table—to achieve exactly‑once semantics in RocketMQ.
This article provides a comprehensive technical overview of large-scale distributed website architecture, detailing its characteristics, goals, layered design patterns, performance and high‑availability optimizations, scalability strategies, extensibility practices, security measures, agile management, and evolutionary case studies for e‑commerce platforms.
This article presents 30 widely‑accepted software architecture principles—grouped into basics, feature selection, server design, distributed systems, user experience, and challenges—to help architects act as gardeners who shape, refine, and sustain robust, maintainable systems.
The article explains why resource isolation is essential in distributed architectures and details various isolation strategies—including thread, process, cluster, data read/write, static, and crawler isolation—illustrated with Netty, Dubbo, Tomcat examples, code snippets, and a comparison of thread‑pool versus semaphore isolation in Hystrix.
This article analyzes the at‑least‑once guarantee of message middleware, explains why duplicate deliveries occur, compares simple database‑based deduplication methods, explores concurrency challenges, and presents a non‑transactional, status‑driven idempotence solution using MySQL or Redis with practical code examples and limitations.
This article analyzes Elasticsearch’s distributed architecture, explaining core concepts such as nodes, indices, shards and replicas, comparing mixed and layered deployment models, and discussing data‑layer design choices and their trade‑offs in modern distributed storage and search systems.
This article explains why message middleware guarantees at‑least‑once delivery, describes three common duplication scenarios in RocketMQ, and presents both transactional and non‑transactional deduplication solutions—including SQL examples and a Redis‑based idempotence library—to achieve exactly‑once processing.
MQProxy is a Java‑based distributed message‑queue proxy built on Apache Kafka that abstracts underlying queue selection, protocols, and health monitoring, offering developers a simple SDK with produce/consume/commit APIs, advanced features like delayed and dead‑letter queues, and a scalable architecture for high availability.
The article explains how Ant Group’s Alipay handles massive double‑11 payment traffic by using logical data centers (LDC), unit‑based architecture with RZone, GZone and CZone, traffic routing, disaster‑recovery strategies, and a CAP‑aware design built on the OceanBase distributed database.
