The article explains how messages travel in monolithic and distributed systems, why network communication is inherently unreliable, and how different delivery semantics—at‑most‑once, at‑least‑once, and exactly‑once—affect reliability, ordering, and protocol choices such as AMQP and MQTT.
This article examines the shortcomings of decentralized and micro‑service‑only approaches for enterprise integration, argues for version‑less services, explains how a centralized ESB and multi‑center architecture can balance security, efficiency, and scalability, and outlines practical design patterns and component roles.
This article provides a comprehensive overview of distributed file systems, covering their historical evolution, essential requirements, architectural models (centralized and decentralized), persistence strategies, scalability, high availability, performance optimization, security mechanisms, and additional considerations such as space allocation, file deletion, small‑file handling, and fingerprint‑based deduplication.
This tutorial walks through building a simple blockchain in Java, covering core concepts, block and transaction structures, proof‑of‑work implementation, and exposing API endpoints via Servlets for creating transactions, mining blocks, viewing the chain, and achieving consensus across nodes.
This article explains the fundamentals of database transactions, the ACID properties, and how they are implemented, then delves into distributed transaction challenges and solutions such as two‑phase commit, three‑phase commit, XA, and Saga patterns, highlighting their trade‑offs and practical usage.
This article details the design, architecture, and implementation of ZanKV—a high‑performance, distributed key‑value store that combines RocksDB storage, etcd‑Raft consensus, and a Redis‑compatible protocol, covering data partitioning, namespace isolation, expiration strategies, cross‑datacenter deployment, and performance tuning.
Zabbix offers a web‑based, enterprise‑grade solution for distributed system and network monitoring; this guide walks Linux ops engineers through why monitoring matters, key availability metrics, what to monitor, step‑by‑step installation, web UI configuration, custom checks, alerting, visualization, template sharing, full‑network scaling, auto‑discovery, proxy deployment, and SNMP integration.
This article provides a comprehensive overview of Spring Cloud, explaining why it’s favored for microservice development, detailing its key features and toolset, and examining essential components such as Eureka, Hystrix, Config, Zuul, and Ribbon with practical insights.
This article provides a comprehensive overview of caching in large distributed systems, covering CDN, reverse‑proxy, local and distributed caches, popular implementations such as Ehcache, Guava, Memcached and Redis, common pitfalls like consistency and avalanche, and practical solutions with real‑world case studies.
Amid rising cloud outages and strict financial regulations, this article examines traditional multi‑active database solutions such as Oracle RAC and IBM GDPS, contrasts them with modern distributed database designs, and details SequoiaDB’s multi‑active architecture and concrete disaster‑recovery procedures for single‑node, site‑wide, and network failures.
This article provides a comprehensive technical overview of Ant Financial’s Service Mesh data‑plane SOFAMosn, covering its architectural background, layered design, core modules, performance optimizations, supported protocols, and future roadmap within a cloud‑native microservices ecosystem.
This article presents a curated knowledge map covering backend architecture, Java best practices, microservice design, distributed consistency, high‑traffic handling, Alibaba frameworks, and design patterns, offering internet high‑concurrency architects a comprehensive reference of essential concepts and skills.
This article explains the fundamentals of database transactions, the ACID properties, the challenges of distributed transactions, and compares various solutions such as 2PC, TCC, local message tables, MQ transactions, and Saga, helping readers choose appropriate approaches based on consistency and availability requirements.
This article presents a chronological overview of Optimistic Concurrency Control (OCC), covering its early theoretical foundations, key research papers, prototype implementations such as MVCC+OCC+2PC and Hekaton, and its adoption in modern distributed NewSQL databases like Megastore, F1, and MaaT, highlighting both advantages and challenges.
This article details Alibaba’s first production use of non‑volatile memory in the Tair MDB cache service, covering deployment, performance challenges such as write imbalance and lock overhead, the optimizations applied, and design guidelines for building NVM‑based caching systems.
Over six years the author chronicles the evolution of a backend system—from its initial three‑day launch, through successive capacity expansions, distributed refactoring, and micro‑service architecture—highlighting the challenges of scaling to millions of users, handling concurrency, and the continuous pursuit of simplicity and resilience.
Elasticsearch is an open‑source, scalable, high‑availability distributed full‑text search engine that operates in near real‑time, using clusters of nodes, indexes, documents, shards and replicas to efficiently store and retrieve large volumes of data.
This article explains how Remote Procedure Call (RPC) underpins micro‑service architectures, connects load balancers like Nginx, powers big‑data frameworks such as Hadoop, fuels AI platforms like TensorFlow, and why mastering RPC is essential for senior backend engineers.
This article traces the step‑by‑step evolution of a web system—from a single‑server LAMP setup through service and data separation, caching, server clusters, read/write splitting, CDN acceleration, distributed databases, NoSQL integration, business splitting, and finally distributed services—highlighting the motivations, characteristics, and practical outcomes of each stage.
This article details the design, architecture, and operational experience of Youzan's Data Platform (DP), covering its scheduling, data‑sync, service, and monitoring modules, the custom Airflow‑based task scheduler, current production metrics, supported task types, and future improvement plans.
This article explores the advantages, architectural layout, network planning, and practical technologies for global server games, emphasizing microservice-based modular design, automated deployment, frame‑synchronization over UDP, centralized versus distributed deployment models, and considerations of latency, data consistency, and blockchain integration.
The article explains the concept of idempotency, why it is critical for data consistency in backend services, and presents practical techniques such as unique business identifiers, locking, optimistic concurrency, deduplication tables, distributed locks, token mechanisms, and payment buffering to achieve reliable idempotent operations.
This article explains the core concepts of distributed architecture, including cohesion and transparency, explores common applications such as distributed file systems, caches, databases, and middleware like MyCat, and outlines the evolutionary stages and challenges of building scalable distributed systems.
This article traces Meituan Dianping's MySQL high‑availability journey, detailing the legacy MMM system, its migration to MHA, integration with Zebra and Proxy middleware, and future architectural ideas such as distributed agents, semi‑sync replication, and MySQL Group Replication.
This article outlines the critical knowledge points for backend developers, covering source code analysis, distributed systems, microservice architecture, performance tuning, Java engineering practices, and comprehensive concurrency programming techniques.
NSQ is a lightweight, high‑performance open‑source message queue written in Go, and this article explains its core components (nsqd, nsqlookupd, nsqadmin, utilities), message flow architecture, configuration details, deployment recommendations, and additional considerations such as persistence, delivery guarantees, and scalability.
This article evaluates the server‑side performance of three popular message middleware platforms—Kafka, RabbitMQ, and RocketMQ—by measuring throughput and latency when sending small 124‑byte messages, revealing that Kafka leads, followed by RocketMQ and then RabbitMQ.
This article introduces ClickHouse as a high‑performance, column‑oriented database designed for real‑time big‑data analytics, outlines its key features, performance characteristics, supported interfaces, differences from Hadoop, and explains its main storage engines—MergeTree and Distributed—while also noting its current limitations.
This article explores the evolution and core concepts of modern distributed architectures—including SOA, microservices, and service mesh—explains key theories such as CAP and BASE, and provides practical guidelines for achieving high availability, scalability, and efficient content delivery through techniques like load balancing, CDN, and gray‑release strategies.
This article provides a step‑by‑step walkthrough of how a Dubbo consumer initiates a remote call, how the request is processed through clustering, load balancing, and invocation layers, and how the provider receives, executes, and returns the result, including the underlying code structures and network interactions.
The article reviews the ten‑year evolution of Ant Financial's message queue, detailing its core reliability, consistency, availability and performance requirements, the architectural mechanisms built to meet them, the shift to pull‑mode and API‑mode designs, and the recent integration of compute capabilities to create a smart data transmission platform.
This article explains core microservice concepts—including componentization, modularization, centralized vs. distributed systems, and the architectural principles behind microservices—while comparing them to SOA and highlighting practical distinctions that help developers design loosely‑coupled, reusable services.
This article explains Kafka's role as a distributed message queue, covering its architecture, replication mechanisms, producer‑consumer workflow, message integrity guarantees, schema management, and performance tuning for high‑throughput, low‑latency backend systems.
This article explains the fundamentals of remote method invocation, compares traditional RMI with modern RPC frameworks, introduces Dubbo's architecture and roles, and provides a step‑by‑step guide to quickly develop, build, and run a complete Dubbo application using API‑centric configuration and multicast registration.
The article explains how the slowdown of Moore's Law drives the shift from monolithic applications to distributed systems, covering load balancing, session handling strategies, read‑write separation, and database sharding to achieve scalable, high‑availability backend architectures.
The article outlines the key characteristics of large-scale websites and traces their architectural evolution from single‑server setups to multi‑tier, cache‑enhanced, clustered, and distributed systems, highlighting strategies such as load balancing, database read/write separation, CDN usage, NoSQL adoption, and service‑oriented decomposition.
This article explains the common misconceptions about software architects, outlines the essential conditions and skills required for senior developers to upgrade, describes practical learning paths, and details the daily responsibilities and interpersonal abilities needed to succeed as an architect.
This article provides a comprehensive technical overview of large distributed website architecture, covering performance, high availability, scalability, security, and agility, and illustrates the evolution, design patterns, and practical optimization techniques for modern e‑commerce platforms.
This article outlines the four major stages of IT architecture evolution—from single‑machine setups, through dual‑machine hot‑standby, multi‑node active clusters, to fully distributed systems—explaining the motivations, challenges, and technologies that drive each transition.
This article introduces core architecture analysis methods—including system engineering, system decomposition, process‑oriented, and object‑oriented approaches—and compares monolithic (ALL IN ONE) and distributed system structures, providing practical examples and visual diagrams.
The article outlines the four major stages of enterprise IT architecture—single‑machine, dual‑machine hot‑standby, multi‑node active‑active, and distributed architectures—explaining their motivations, advantages, limitations, and how businesses should choose the appropriate model based on performance, availability, and scalability requirements.
This article details Meituan's practical approaches to boosting account service reliability, covering MTBF/MTTR metrics, business‑level monitoring, flexible availability with circuit‑breaker patterns, cross‑region active‑active deployment, data synchronization techniques, and the measurable performance gains achieved.
This article explains how Weibo handles massive daily traffic of over a hundred billion requests by employing a five‑layer feed system and a six‑layer cache architecture that evolved from simple KV storage to sophisticated counter and existence‑check services, highlighting design choices, performance optimizations, and future directions.
This article traces the historical development of distributed architecture from early computers to modern large‑scale web systems, explains core concepts such as clustering, replication, and middleware, and outlines eight evolutionary stages that address scalability, reliability, and performance challenges.
The talk details Tencent Xinge’s architecture and cloud‑native practices that enable hundred‑billion‑level mobile push, combining terminal integration, real‑time backend filtering, distributed bitmap selection, precise‑push AI models, and DevOps pipelines to deliver fast, scalable, data‑driven notifications with effect tracking.
The Mi Master billing platform, a SaaS service from Tencent that processes over 735 billion RMB quarterly, provides a unified, modular architecture with distributed multi‑master databases, high‑availability cross‑region deployment, multi‑stage fraud detection, and global support for 80+ payment channels across 180+ countries, delivering seamless APIs, automated reconciliation, and extensive revenue‑sharing tools for products such as Honor of Kings, PUBG, WeChat Pay, and QQ Wallet.
This talk details SF Express’s journey from heterogeneous legacy databases to standardized open‑source, distributed architectures and intelligent operations, covering standardization, migration to open‑source, scaling with Mycat, automated resource pooling, and the ThinkDB platform that drives proactive, automated DBA workflows.
This article examines common hardware and network failures in data centers, analyzes real‑world outage cases, classifies fault domains, and presents comprehensive strategies for database fault handling—including logging, checkpointing, backup, replication, and high‑availability architectures—to improve reliability and reduce downtime.
This article examines three common cache‑update approaches—updating the cache after the database, deleting the cache before updating the database, and updating the database then deleting the cache—analyzes their drawbacks, and presents practical solutions such as delayed double‑delete and retry mechanisms to ensure data consistency.
Senior Alibaba engineer Xiao Xie recounts his decade‑long journey from the early Taobao Mall project to leading the Cloud Computing “Flying‑Sky Eight” team, detailing pivotal initiatives like the Five‑Color Stone integration, full‑link stress testing for Double‑11, and the evolution toward self‑developed, distributed cloud technologies.
This article explores the essence of system architecture for massive web services, illustrating strategic and tactical considerations through examples like Uber and Weibo, and discusses key capabilities such as abstraction, classification, performance, service decomposition, multi‑level caching, distributed tracing, and continuous learning for scalable backend design.
The article recounts how Alibaba and Ant Group were forced by the massive Double‑11 transaction surge to develop their own OceanBase database, explores the five‑year struggle, the broader fintech innovations, and the strategic mindset of autonomous R&D and open‑source sharing.
The article details Minsheng Bank’s shift to distributed architecture, outlining regulatory drivers, business requirements, the adoption of sharding, cross‑center high‑availability, and new‑type distributed databases, and showcases performance results of SequoiaDB 3.0 across multiple high‑throughput banking scenarios.
This article examines Alipay's evolution from a simple single‑datacenter setup to a multi‑active‑active, unit‑based architecture, detailing the technical challenges of high availability, disaster recovery, failover design, blue‑green deployment, and how these solutions enable continuous service during massive traffic spikes like Double 11.
This article examines the importance of high‑availability and disaster‑recovery architectures, tracing Alipay’s evolution from a simple load‑balanced setup through multi‑datacenter, failover, and unit‑based designs that address scalability, data consistency, and continuous service delivery challenges.
This article explains the fundamentals of Raft and PBFT consensus algorithms, compares their processes, fault tolerance limits, and suitability for private and consortium blockchains, and highlights key differences in leader election, message handling, and fault models.
This comprehensive note distills the evolution of large‑website architecture—from single‑server setups to layered, distributed, and highly available systems—covering caching, clustering, read/write separation, CDN, NoSQL, business splitting, scalability, extensibility, and automation strategies.
This article summarizes the evolution stages, architectural patterns, and key concerns such as performance, scalability, extensibility, high availability, and distributed design that large‑scale websites must address, providing practical insights and visual diagrams for each concept.
This article surveys the evolution of schedulers—from early batch systems and OS process schedulers to modern centralized, two‑level, and shared‑state designs—explaining their core concepts, trade‑offs, and real‑world examples such as YARN, Mesos, Spark, Borg, and Kubernetes.
This article explains the essence of software architecture as a process of reducing system entropy through splitting and merging, outlines the capabilities required of architects, and details the step‑by‑step evolution of large‑scale website infrastructures including caching, CDN, database sharding, and messaging systems.
This article explains the design and evolution of WeChat's high‑availability seqsvr service, which provides per‑user 64‑bit sequence numbers for data synchronization, handling trillions of requests with millisecond latency through pre‑allocation, section sharing, and layered storage architecture.
This article outlines practical strategies for building high‑concurrency back‑end systems—including server architecture, load balancing, database clustering, caching, message queues, asynchronous processing, and service‑oriented design—to ensure smooth operation of traffic‑intensive e‑commerce services.
This article introduces Dubbo’s background, explains the evolution of e‑commerce architectures to RPC‑based distributed systems, details Dubbo’s components, advantages, and drawbacks, and provides a complete Maven‑based demo—including Zookeeper installation, Spring configuration, and Java code—for building and consuming a Dubbo service.
Meituan Waimai handles ten‑million daily orders by evolving from a tiny monolithic prototype to a distributed, micro‑service‑based platform that uses sharded databases, caches, set‑based traffic partitioning, automated AIOps, dynamic container scaling, prioritized degradation switches, and AI‑driven features to sustain massive, growing traffic.
The article explains the concept of distributed architecture, its key characteristics such as cohesion and transparency, showcases how MySQL and middleware like Mycat are used in e‑commerce platforms, and outlines the evolution, practical implementations, and challenges of building scalable distributed database systems.
This article outlines essential distributed system concepts—including system decomposition, concurrency, caching strategies, online vs. offline processing, push/pull communication, load limiting, service degradation, CAP theorem, and eventual consistency—to help engineers design scalable, reliable architectures for high‑traffic applications.
This article demystifies reliability by clarifying the difference between durability and availability, exposing common misconceptions about MTBF, analyzing real‑world disk failure data, and presenting a practical formula for calculating the health probability of distributed storage systems.
This article examines core distributed system design principles—reliability, scalability, and maintainability—explaining how techniques such as replication, partitioning, consensus algorithms, and transactions address hardware, software, and human failures, and discusses vertical and horizontal scaling strategies to achieve robust, extensible, and maintainable architectures.
This article explains the architecture, data model, and multi‑tenant load‑balancing strategies of Alibaba Cloud Table Store, detailing the challenges of distributed NoSQL systems and presenting practical solutions for resource quantification, fairness, trigger timing, and SLA‑driven automation.
This article examines the limitations of traditional microservice frameworks, introduces service mesh as a solution with sidecar architecture, outlines its definition, evolution stages, and timeline, and concludes with resources for further learning and practical implementation.
This article reviews the challenges of early microservice frameworks—high technical barriers, limited multi‑language support, and intrusive code—and explains how service mesh architectures with sidecar proxies, exemplified by Linkerd, Envoy, and Istio, provide a dedicated, language‑agnostic infrastructure layer that simplifies service governance and operations.
This article details how Ele.me engineered a cross‑region active‑active system that scales elastically, tolerates whole‑data‑center failures, and maintains real‑time food‑delivery performance through geographic sharding, intelligent routing, and robust data‑replication middleware.
Milano, a distributed log collection and analysis platform built on the ELK stack, leverages Filebeat, Kafka, Logstash, Elasticsearch, and Kibana to provide high‑throughput, low‑latency, secure, and visual log management for massive clusters, addressing the challenges of traditional manual log inspection.
In his Money 20/20 Asia keynote, Ant Group CTO Cheng Li outlines the company’s fintech roadmap, highlighting AI‑driven risk engines, blockchain‑based trust mechanisms, cloud‑native infrastructure, and innovative mobile payment solutions that aim to make financial services more inclusive and efficient.
This article explains the concept of high concurrency, outlines a server architecture that supports it—including load balancing, distributed databases, NoSQL caches and CDN—and presents practical testing methods and implementation patterns such as caching strategies and message‑queue designs to handle massive simultaneous requests.
The article details Alipay’s comprehensive architecture for the Double 11 shopping festival, covering its three‑layer IAAS/PAAS/SAAS model, logical data‑center design, multi‑active disaster‑recovery, blue‑green deployment, distributed data sharding, transaction processing, and the Ant Credit Pay service’s performance and risk‑control mechanisms.
This article discusses the importance of source code analysis, outlines key concepts in distributed systems, explains microservice architecture, highlights performance optimization techniques for Java applications, and presents practical engineering advice for modern backend development.
This article explores the importance of data consistency in distributed systems, illustrating real‑world scenarios, explaining consistency models such as strong, weak and eventual, and detailing the challenges and theories like CAP and BASE that guide system designers in balancing consistency, availability, and partition tolerance.
This article examines Baidu’s Cluster Control System (CCS), detailing its two‑level data model, four‑tier scheduling architecture, and three‑layer execution agents, and explains how control and execution information, redundancy, and fault‑tolerant designs enable reliable large‑scale command execution across thousands of servers.
This article introduces message queue middleware, explains its role in distributed systems for decoupling, asynchronous processing, traffic shaping, and log handling, and reviews typical use cases, architectural patterns, JMS models, and popular products such as ActiveMQ, RabbitMQ, ZeroMQ, and Kafka.
This article explores the fundamentals of command execution, examines the challenges of scaling command delivery across hundreds of thousands of servers, and details Baidu’s Cluster Control System architecture that enables efficient, flexible, and extensible distributed command management for operations teams.
This article explains the limitations of simple modulo hashing for Redis clusters, introduces consistent hashing with a virtual‑node ring to achieve fault tolerance and seamless scaling, and demonstrates how the algorithm reduces data skew and improves cache performance in distributed systems.
This article explains the circuit breaker pattern for distributed systems, detailing its problem context, state machine solution, implementation in C#, key considerations, usage scenarios, and comprehensive unit tests, illustrating how to improve system resilience and prevent cascading failures.
This article reviews cloud gaming resource management, introduces search‑engine instance distribution techniques, explores AI‑based disk‑failure prediction and load forecasting, and presents replica and DDoS‑detection strategies to improve efficiency and reliability of large‑scale distributed systems.
In this extensive interview, senior Alibaba engineer Chen Kangxian shares his experiences designing large‑scale distributed systems, live‑streaming platforms, and high‑concurrency architectures, offering practical insights on technology choices, failure handling, and career growth for software architects.
This article introduces Hulu's in‑house OLAP engine Nesto, detailing its near‑real‑time data ingestion, nested data model, TB‑level storage using HBase and Parquet, MPP query execution, custom predicate library, and the overall architecture that enables sub‑second ad‑hoc queries for user analytics.
This article outlines ten fundamental architecture patterns for high‑traffic websites—including layering, partitioning, distribution, clustering, caching, asynchronous processing, redundancy, automation, and security—explaining their goals, benefits, challenges, and best‑practice constraints to help engineers build scalable, reliable, and maintainable systems.
This article explains Elasticsearch’s cluster architecture, detailing node roles, index, shard, and replica concepts, compares mixed and tiered deployment models, examines data storage mechanisms, and discusses the advantages and trade‑offs of different distributed system designs.
This article explores the defining traits of large‑scale websites and walks through the step‑by‑step evolution of their architecture—from single‑server setups to distributed systems with caching, load balancing, database sharding, and micro‑services—while highlighting common design pitfalls and best‑practice recommendations.
Facing billions of rows spread across 1024 tables in 64 databases, a Chinese gaming platform replaced its custom sharding framework with TiDB, achieving real‑time OLTP consistency, elastic scaling, robust backup, high availability, and lower migration costs while preserving MySQL compatibility.
Explore how to configure Celery’s periodic (Beat) tasks, perform synchronous task calls, persist results using Redis, monitor workers with Flower, and debug remotely via telnet, with practical code examples and step‑by‑step instructions for robust backend task management.
This article explores how to design a high‑performance, highly available, and scalable e‑commerce platform by separating business and technical architectures, detailing subsystem decomposition, scaling strategies, and the evolution from simple single‑server setups to distributed, clustered solutions.
Meituan designed a Storm‑based real‑time data processing platform that guarantees at‑least‑once delivery and high availability, employs a custom spout, regression‑driven traffic smoothing, and a low‑latency KV store with atomic operations, persisting results in Kafka, MySQL and Cellar to power merchant dashboards and heat‑tag analytics, while planning broader real‑time analytics expansion.
The article critically examines micro‑services, outlining their promised benefits such as independent development, deployment and scaling, while exposing the hidden operational, dev‑ops, state‑management, communication, versioning and distributed‑transaction challenges that can turn them into a fragile, overly complex system.
This article explains the microservices architecture style, compares it with monolithic and SOA approaches, outlines design principles, communication mechanisms, data decentralization, integration patterns, and discusses the advantages and disadvantages of adopting microservices in modern software systems.
A staging environment’s new scheduled task kept failing due to Zookeeper disconnections caused by packets exceeding the default 1 MB maxBuffer, and the article explains the root cause, heartbeat timing, and how adjusting Djute.maxbuffer or upgrading Zookeeper resolves the issue.
Distributed systems rely on consensus to ensure consistent results, but achieving it faces fundamental challenges such as network unreliability, node failures, and trade‑offs captured by the CAP theorem, FLP impossibility, and various algorithms like Paxos, Raft, and Byzantine Fault Tolerance, each balancing consistency, availability, and safety.
This article explains the core challenges of achieving consistency in distributed systems, describes consensus algorithms such as Paxos and Raft, discusses theoretical limits like the FLP impossibility and CAP theorem, and shows how trade‑offs among consistency, availability, and partition tolerance shape practical system design.
This article explains the essential concepts of blockchain testing, covering its unique characteristics, testing difficulties, required technical knowledge, key test scenarios, performance metrics, and future directions toward automated and unified testing across industries.
Facing over 60 million daily users and 100 K QPS, iQIYI’s Bubble platform migrated from a monolithic codebase to a business‑driven microservice architecture—splitting services by entity and function, adopting the internal RPCHUB RPC framework, establishing ownership, fault‑tolerance, monitoring and CI/CD pipelines, and addressing scaling challenges to sustain rapid growth.
