This article explains how to design a robust, zero‑intrusion data fallback component for high‑traffic web services using Nginx, Lua, and AOP principles, covering problem definition, architectural options, detailed execution steps, configuration snippets, storage choices, and performance monitoring.
This article demonstrates how to enable Oracle Data Guard’s Maximum Protection mode, examines the behavior of primary and standby databases when the standby connection is broken, and walks through the resulting errors, forced shutdown, and step‑by‑step recovery back to normal operation.
Meituan’s food‑delivery order platform evolved from a simple monolithic prototype in 2013 to a distributed, highly available service that now handles tens of millions of orders daily, using asynchronous processing, sharding, multi‑data‑center deployment, and automated operations to achieve sub‑150 ms transaction latency and 99.9999% uptime.
This tutorial shows how to create a three‑node MongoDB replica set using Docker, configure automatic primary election, verify data synchronization across secondaries, and simulate a primary failure to observe failover, providing a practical guide for high‑availability database deployments.
This article examines common misconceptions in designing geo-distributed active-active systems, explains why striving for perfect real-time data sync is unrealistic, and offers practical strategies—such as prioritizing core services, reducing distance, limiting data replication, and combining storage sync with messaging—to achieve reliable high-availability.
The article reviews WeChat’s newly open‑sourced PhxSQL, a MySQL clustering solution that promises high availability and strong consistency, examines its complex architecture with added Phxbinlogsvr and Phxsqlproxy components, and discusses practical concerns such as deployment difficulty, replication latency, lack of multi‑master writes, semi‑synchronous behavior, and monitoring challenges.
This guide walks through setting up a high‑availability HAProxy and Keepalived dual‑master configuration, detailing the experiment environment, installation steps, keepalived scripts, and testing procedures that ensure continuous service even if one node fails.
PhxSQL is an open‑source, MySQL‑compatible relational database cluster that uses a Paxos‑based log store and a three‑layer architecture (Phxbinlogsvr, Phxsqlproxy, Phxsync) to provide automatic master failover, strong data consistency, and improved write performance compared to MySQL semi‑synchronous replication.
The article details Meizu's Flyme operations platform evolution—from a single‑cabinet setup in 2011 to a multi‑IDC, 6000‑server infrastructure—highlighting challenges, architectural upgrades, monitoring, cost control, automation, and future high‑availability directions for large‑scale internet services.
The article details how Weibo’s cache infrastructure progressed from simple Memcache deployments to a sophisticated, service‑oriented architecture featuring multi‑layer caching, proxy layers, dynamic configuration, monitoring, and automated scaling to meet massive read‑write demands and high availability requirements.
This article examines MySQL's semi‑synchronous replication, analyzes consistency scenarios across master‑slave failures and switches, and highlights the difficulties of data rollback and multi‑master conflicts that affect high‑availability clusters.
This article examines common MySQL high‑availability designs—including single master‑slave, dual‑master with multiple slaves, Heartbeat + DRBD, MySQL Cluster, semi‑synchronous replication, and GTID—explaining their benefits, drawbacks, and practical considerations for reliable failover and scalability.
This talk explains how Lianjia redesigned its MySQL high‑availability setup by replacing VIP‑based failover with a Zookeeper‑driven naming service, detailing the original MHA architecture, its shortcomings, the new components, workflow, operational challenges, and a DNS‑based refinement.
This article explains the comprehensive concept of high‑availability systems, covering redundancy, failover, consistency challenges, various technical solutions, SLA definitions, and the organizational and engineering practices required to achieve multiple “9s” of availability.
The article details the evolution, system architecture, backend technologies, high‑availability designs, data handling, upload and playback workflows, and operational monitoring of the Miaopai short‑video service, illustrating how it scales to hundreds of millions of users under heavy load.
This tutorial walks through setting up a MySQL master‑slave replication cluster, configuring password‑less SSH, installing MHA‑node and MHA‑manager, creating the MHA configuration, validating the setup, and testing automatic failover when the master goes down.
The article presents a detailed overview of Bilibili's GOIM live chat architecture, covering its high‑stability, high‑availability, low‑latency design, component breakdown, memory and module optimizations, network improvements, and performance testing results to achieve scalable real‑time messaging.
This article details a MySQL high‑availability solution built on Pacemaker, Corosync and MHA, explains why earlier keepalived‑based designs suffered split‑brain issues, and walks through the architecture, quorum handling, resource agents, failover workflow, testing methodology, and practical lessons learned.
This article demonstrates how MaxScale manages MySQL read/write splitting when one or both slave servers fail, showing log configurations, state changes, and how to configure a stable master to maintain service continuity even when all slaves are down.
MHA (Master High Availability) ensures MySQL cluster resilience by monitoring the master, automatically promoting the most up‑to‑date slave as a new master during failures, offering fast failover, strong data safety, easy extensibility, and multi‑cluster management, while noting its limitations.
This tutorial walks through preparing a CentOS 6.6 environment, creating MHA users and SSH keys, configuring hosts, installing MHA Node and Manager, verifying configurations, and testing master failover to achieve reliable MySQL high‑availability using MHA.
The article describes how Alibaba's game integration platform achieved business‑oriented high availability by abandoning traditional system‑centric designs and implementing a three‑dimensional architecture that combines clear HA goals, multi‑active deployment, client‑side retries, functional isolation, automated monitoring, and rapid fault recovery, ultimately meeting a 3‑minute issue‑location and 5‑minute business‑recovery target.
This guide walks through configuring a Percona XtraDB Cluster on three OpenStack nodes, disabling SELinux and firewalls, installing and tuning MySQL, setting up HAProxy for load balancing, deploying Keepalived for failover, and verifying high‑availability with practical test steps.
This article provides a detailed walkthrough of MHA's core configuration parameters—including server scopes, connection settings, candidate master rules, failover scripts, and monitoring options—explaining where each setting belongs and how to fine‑tune MySQL high‑availability behavior.
This article explains how MHA provides automated master monitoring, rapid failover within 10‑30 seconds, online master switching, various failover modes, extensible scripts, key commands, and best‑practice workflows to achieve high‑availability MySQL replication without performance impact.
This article examines how to maintain game world state using data services, discussing traditional DB proxy processes, the role of Redis as a cache and database, stateless versus stateful services, sharding, high‑availability mechanisms such as Sentinel, Zookeeper, and the overall architecture for scalable MMO back‑ends.
This guide walks through MySQL high‑availability architectures, comparing failover solutions like MMM and MHA, detailing MHA installation, configuration, and testing, and exploring MaxScale's read/write splitting, routing strategies, and integration with MHA for seamless failover.
This article explains MHA’s architecture, failover workflow, comparison with other MySQL HA solutions, and its six key advantages, showing how it can switch masters within seconds while preserving data consistency without altering MySQL settings or adding many servers.
This article collection reviews essential architectural concepts—including scalable web development, business‑centric high‑availability design, front‑end/back‑end separation with REST, large‑scale instant‑messaging platforms, industrial IoT frameworks, software‑defined storage, and the personal growth required of architects—providing practical guidance for building robust, adaptable systems.
This guide walks through constructing a Kubernetes master high‑availability environment on a private‑cloud Ubuntu 14.04 platform, covering full component containerization, external load‑balancer setup with HAProxy and Keepalived, static pod configuration, privileged mode requirements, flannel networking, common pitfalls, and future HA master developments.
This article explores typical cache challenges in distributed systems—including data consistency, high availability, cache avalanche, and cache penetration—explaining their causes, real‑world scenarios, and practical mitigation strategies to ensure reliable and efficient caching.
The article explains how modern call centers evolve to multi‑site active‑active deployments, detailing the network design, disaster‑recovery mechanisms, login and monitoring logic, key features supporting automatic failover for thousands of agents, and future expansion possibilities.
The article describes how a large‑scale call center evolves to a multi‑site architecture and implements system‑level active‑active disaster recovery using Ctrip's contact‑center and unified login platforms, detailing the login flow, fault‑detection logic, key features, and future extensions.
This article provides a comprehensive overview of software load balancing, detailing the architecture, features, performance, and deployment strategies of Nginx, LVS, and HAProxy, and includes practical scenarios for front‑end, internal, and high‑availability setups.
This article explains how load balancing solves high‑traffic, high‑concurrency challenges by distributing requests across multiple servers, covering its core principles, vertical and horizontal scaling, various classification methods such as DNS, IP, link‑layer, and hybrid approaches, along with their advantages, drawbacks, and deployment recommendations.
This article explains the core distinctions between distributed systems and clusters, covering their architectures, efficiency goals, typical use cases, and examples such as Hadoop MapReduce and load‑balancing clusters, while also detailing cluster types, high‑availability, load balancing, and high‑performance computing.
This article explores microservice architecture fundamentals, compares synchronous and asynchronous communication, details RabbitMQ’s AMQP model, optimization techniques, high‑availability configurations, flow‑control mechanisms, and shares practical case studies from NetEase’s Hive platform, offering actionable insights for reliable, scalable message‑queue deployments.
The article proposes a cross‑IDC disaster‑recovery architecture for key‑value stores that uses a storage proxy layer to record write logs in the primary data center, forward them to a read‑only backup center for replay, and route reads via load‑balancing, while noting current limitations such as complex configuration, low synchronization concurrency, and lack of multi‑write support.
Redis replication offers high‑availability through master‑slave syncing, supporting multiple slaves, cascading replication, non‑blocking data transfer, read‑write separation, and optional disk‑less sync, with detailed steps for configuration, safety considerations, partial resynchronization, and practical deployment examples.
The article shares practical experiences and lessons learned from three large‑scale backend architecture refactorings, explaining why doing business‑critical changes while keeping services online is the toughest challenge and how to identify and solve the most critical problems first.
The article reviews common MySQL high‑availability solutions—including shared‑storage SAN, DRBD disk replication, keepalived/heartbeat, MHA, ZooKeeper‑based HA, Galera/PXC clusters, and proxy middleware—detailing their architectures, advantages, limitations, and suitability for different business and operational requirements.
This article explains the fundamental differences between distributed systems and clusters, illustrates them with real‑world examples such as Dubbo, Elasticsearch and Git, and details various cluster types, high‑availability and load‑balancing strategies, as well as fault‑tolerance mechanisms.
This guide compares Percona Server, MariaDB, and MySQL, outlines practical MySQL performance tuning steps across hardware, storage, filesystem, and server parameters, and presents several high‑availability architectures such as master‑slave replication, MMM/MHA, Heartbeat/SAN, and Heartbeat/DRBD with real‑world deployment examples.
This article provides a comprehensive overview of load balancing, covering its core principles, the distinction between vertical and horizontal scaling, various classifications such as DNS, HTTP, IP, link‑layer, and hybrid approaches, as well as common algorithms and hardware considerations for high‑availability, high‑performance systems.
This article explains how to build a micro‑blogging social system using Redis, covering core data models, read‑through replication, Sentinel failover, and write‑side sharding techniques that enable the platform to handle hundreds of millions of users and massive traffic.
This article presents a practical dual‑master MySQL high‑availability solution using keepalived and custom shell scripts that reliably detect master health, perform graceful VIP failover, and safely switch back after repairs, while ensuring data consistency and minimal downtime.
This guide explains how to use Keepalived and the VRRP protocol to build a two‑node high‑availability setup for Nginx, covering installation, health‑check scripts, configuration options, testing procedures, and troubleshooting tips for seamless failover.
This article explains the design principles, architecture, and performance optimizations of Alibaba's Notify and MetaQ message middleware, illustrating how they achieve reliable asynchronous communication, high scalability, and low latency for billions of messages during peak e‑commerce events like Double 11.
This article provides an in‑depth technical overview of Meizu's cloud sync service, covering its business scenarios, the evolution of its core SyncML‑based protocols, modular architecture across multiple data centers, performance optimizations such as traffic reduction and connection handling, as well as high‑availability, security, and future SDK‑API extensions.
This article explains IBM PowerHA (formerly HACMP) for AIX, covering its architecture, heartbeat‑based failover mechanisms, topology and resource components, dual‑active HyperSwap solutions, and the role of the HMC management console in providing enterprise‑grade high availability.
This article examines how a large‑scale e‑commerce transaction platform achieved high availability through iterative architectural evolution—from early .NET monoliths to vertical and horizontal micro‑service splits—highlighting practical strategies for fault detection, rapid recovery, scaling, and operational best‑practices.
This step‑by‑step guide shows how to configure a three‑node MariaDB HA cluster on CentOS using NFS shared storage, corosync multicast heartbeat, pacemaker resource management, and a virtual IP, including installation, configuration files, and failover testing.
This article provides a comprehensive technical summary of large‑scale distributed website architecture, outlining characteristics, architectural goals, patterns, and detailed strategies for high performance, high availability, scalability, extensibility, security, and agility, supplemented with illustrative examples and practical insights.
The article provides a comprehensive overview of IBM's General Parallel File System (GPFS), detailing its physical and logical architecture, key components such as NSD and quorum mechanisms, scalability, load balancing, and fault‑tolerance features for parallel and serial applications.
This guide details the complete process of moving a 2‑node Oracle 11gR2 RAC database to new servers and storage using Active Physical Standby Data Guard, covering architecture, pre‑implementation preparation, detailed implementation steps, switchover procedures, impact assessment, rollback measures, and lessons learned.
This article introduces Linux Virtual Server (LVS), explains its integration into the Linux kernel, outlines its goals for high‑performance, highly available server clusters, and provides a brief mind‑map overview along with a quiz on LVS modes and its founder’s nickname.
Freeman, a 1988‑born O2O senior operations engineer from Henan now in Shanghai, shares his personal learning saga—from clueless newcomer to mastering Linux, Nagios, iptables, load‑balancing, MySQL replication, and high‑availability architectures—highlighting the challenges, training, and mindset needed to thrive in modern IT operations.
The article details Upyun's high‑availability image processing architecture, covering workload‑aware system design, custom GmServer implementation, task scheduling, current strengths and limitations, and future directions such as a ServiceServer‑based queue and Docker‑driven dynamic scaling.
The article explains how multi‑datacenter architectures prevent total service loss, improve latency by placing services near users, and balance the CAP trade‑offs through models like AC, CP, and AP, while outlining practical design, sharding, monitoring, and failover strategies for large‑scale backend systems.
Before migrating enterprise OLTP workloads to open‑source MySQL, this guide evaluates three high‑availability architectures—Percona XtraDB Cluster, MHA, and native dual‑master—by outlining key pre‑migration questions, their advantages, drawbacks, and practical deployment considerations.
The article outlines how Meipai’s rapidly growing platform built a backend system that guarantees high availability, handles massive traffic, supports evolving media formats, and adopts a multi‑language development model with clear principles for language selection and systematic integration.
This article provides a comprehensive overview of VMware's high‑reliability PSA architecture, virtual machine conversion tools, HA, DRS, FT, vMotion, storage vMotion, DPM, vNetwork distributed switch, and automation utilities, explaining their functions, components, and operational benefits within virtualized environments.
This article examines the design considerations of real-time distributed streaming systems, outlines their background and characteristics, compares the architectures of Spark Streaming and Storm, discusses primitives, message passing, high availability, storage models, and integration with production environments, providing practical insights for architects.
Kafka introduced a high‑availability mechanism in version 0.8 by replicating partitions across multiple brokers, designating a leader and followers, using an in‑sync replica (ISR) list to balance synchronous and asynchronous replication, and employing leader election strategies to maintain data integrity during failures.
Load balancing distributes incoming traffic across multiple servers to enhance performance, ensure high availability, and enable horizontal scaling, with various implementations such as DNS, HTTP, IP, link‑layer, and hybrid methods, each employing algorithms like round‑robin, random, least connections, hash, and weighted distribution, plus hardware options.
The article analyzes the architecture of TDSQL, a MySQL‑based distributed SQL system designed for Tencent’s billing platform, detailing its transition from an in‑memory NoSQL solution to a high‑consistency, auto‑scaling, fault‑tolerant database with sharding, scheduler, agent, and gateway components.
This article explains how Booking.com solved MySQL master‑to‑hundreds‑of‑slaves bandwidth limits by introducing a Binlog Server component, detailing its design, advantages over traditional relay‑master setups, and multiple deployment scenarios for high‑availability and cross‑region replication.
The article reviews eight expert presentations from the 2015 SDCC Database Practice Forum, covering distributed database architectures, performance tuning, high‑availability solutions, and practical case studies from leading Chinese internet companies.
The article presents the Binlog Server, a component that replaces intermediate masters in MySQL replication topologies to reduce network saturation, simplify disaster recovery, and improve high‑availability while supporting large numbers of slaves across remote sites.
This article explores the importance of database clustering in modern information systems, outlines the challenges of performance, availability, and scalability, and compares Share‑Disk and Share‑Nothing architectures along with their advantages, drawbacks, and real‑world implementations.
During the Chinese New Year, Tencent Cloud handled billions of WeChat red‑envelope transactions by employing multi‑region load balancing, disaster‑recovery architectures, database sharding, and high‑throughput caching to ensure seamless, high‑availability service for millions of concurrent users.
This article outlines essential design principles for large‑scale web architectures—including availability, performance, reliability, scalability, manageability and cost—and demonstrates their application through a detailed image‑hosting service example, covering services, redundancy, partitioning, caching, proxies, indexing, load balancing, and queuing to achieve efficient, scalable data access.
This article shares practical insights on achieving high availability by understanding goals, decomposing requirements, designing resilient architectures, ensuring operability, testing rigorously, and reducing release risk, illustrated through the multi‑stage evolution of a transaction system.
The article outlines the design requirements, sharding strategies, capacity planning, cost trade‑offs, and indexing techniques for building a highly available, scalable distributed storage solution for Weibo, focusing on storage‑layer decisions without involving caching or ID generation.
This article outlines Meipai’s evolution into a billion‑user short‑video platform, detailing its high‑availability, scalable architecture, service discovery via etcd, data storage challenges, CDN and cloud‑storage redundancy, fault‑tolerance mechanisms, and future directions such as H.265 adoption and P2P‑CDN hybrid delivery.
This technical session covers Oracle RAC high‑availability best practices, installation steps, daily operational commands, detailed case studies of auto‑start checks, version‑mix issues, addNode failures, network heartbeat problems, and client connection errors, plus a concise Q&A on uninstall, SCAN vs VIP, and split‑brain detection.
This article shares the architecture, design principles, challenges, and performance‑optimizing solutions behind a ten‑hundred‑thousand‑scale server monitoring system, covering data collection agents, distributed pipelines, real‑time alerts, high throughput, multi‑platform support, and practical lessons learned.
The article details Weibo's journey in building a multi‑data‑center active‑active architecture, covering its evolution, technical challenges such as latency and data synchronization, the adopted MCQ‑based messaging solution, operational best practices, and future directions for high‑availability deployments.
This article explains MySQL replication fundamentals, detailing master‑slave and multi‑master architectures, thread roles, configuration parameters, read‑write splitting, multi‑source setups, and advanced scenarios such as multi‑level and circular replication, while highlighting common pitfalls and performance considerations.
Facing up to 1.6‑1.7 billion daily requests, Momo’s tech director explains how the company built a highly available, low‑latency communication protocol and distributed infrastructure—leveraging virtualization, OpenStack, and cloud servers—to overcome weak mobile networks and serve its massive social‑gaming user base.
The article explains 58.com’s service‑oriented architecture that combines dual‑master MySQL replication, a standby database, and Redis caching to provide high‑availability read/write operations, outlines their scaling process, and details a four‑step cache‑consistency strategy.
This article describes JD’s unified service architecture for the product detail page that handled tens of billions of requests during Double Eleven, detailing the multi‑level caching strategy, Nginx+Lua access layer, service isolation, degradation mechanisms, and front‑end logic offloading.
This article walks through implementing web high availability and static‑dynamic content separation using HAProxy combined with Keepalived, covering load‑balancing concepts, VRRP basics, step‑by‑step configuration of time sync, hostnames, SSH trust, installing required packages, and testing failover scenarios.
This article explains the design, storage mechanisms, high‑availability architecture, scaling strategies, multi‑data‑center disaster recovery, operational management, and future plans of Taobao’s distributed file system (TFS), a highly available and scalable storage solution for massive unstructured data.
Taobao’s File System (TFS) is a highly available, high‑performance distributed storage solution built on Linux servers, featuring name‑server and data‑server clusters, block‑level replication, HA mechanisms, client caching, seamless scaling, multi‑data‑center disaster recovery, and open‑source support for C++, Java, and Nginx integration.
The article describes how the limitations of an in‑memory NoSQL HOLD platform led to the creation of TDSQL, a distributed MySQL‑based SQL system featuring ZooKeeper‑coordinated Scheduler, Agent, and Gateway modules, automatic cross‑IDC disaster recovery, transparent horizontal scaling, strong synchronous replication, and future integration with container technologies.
The article chronicles how 58.com’s web architecture evolved from a tiny, single‑machine setup to a multi‑layer, Java‑based, highly available service‑oriented platform, detailing the technical decisions, scaling challenges, and automation practices adopted at each traffic milestone.
This article outlines the UIOC (Urgent Incident Office Center) process, detailing a six‑step workflow, multi‑team collaboration tactics, common troubleshooting methods, and pitfalls to avoid, helping operations teams achieve 99.99% availability while handling emergencies efficiently.
Tencent’s operations team explains how its SET‑based NoSQL clusters deliver ultra‑low latency, high availability, and seamless disaster recovery for billions of users, detailing deployment models, synchronization mechanisms, cost‑saving techniques, and the Data‑as‑Service approach that underpins its massive social platforms.
This article explains how TDSQL transforms MySQL into a distributed, high‑availability SQL system by addressing NoSQL limitations, introducing a Scheduler‑Agent‑Gateway architecture, automatic scaling, sharding, robust disaster‑recovery mechanisms, and future integration with container technologies.
This article details the evolution and redesign of JD.com’s product detail page architecture, describing the transition from static HTML generation to a dynamic, high‑performance, multi‑datacenter system built on key‑value storage, Nginx + Lua, asynchronous processing, multi‑level caching, and robust scaling and reliability strategies.
This article describes how JD.com redesigned its product detail page system from a static, cache‑heavy architecture to a fully dynamic, multi‑level cached service using Nginx+Lua, JIMDB, and asynchronous workers, addressing scalability, performance, and high‑availability challenges for billions of daily page views.
The article analyzes the use of a master/slave architecture for distributed caching, explains why two clusters, CAS, and proxy are employed, discusses consistency and availability challenges, and evaluates possible mitigation strategies for cache failures.
DRBD (Distributed Replicated Block Device) is a software‑based, network‑driven block replication solution that mirrors disks, partitions, or logical volumes across servers in real time, offering synchronous and asynchronous modes, transparent failover, and a middle‑layer between the filesystem and physical storage.
The article explains how distributed systems split a task into multiple subtasks across separate servers to reduce execution time, while clusters replicate the same service on many servers to increase throughput, highlighting their complementary roles in scalability and high availability.
This article explains how Keepalived automatically monitors web servers at the IP, TCP, and application layers, removes unhealthy nodes, and reintegrates them once recovered, thereby maintaining high availability and reducing manual maintenance in complex server clusters.
When read traffic overwhelms a single MySQL master, cascading replication and dual-master setups can distribute slave load, reduce replication lag, and simplify failover, offering a scalable solution that balances read/write pressure across multiple replication layers.
An in‑depth look at Weibo’s evolution to its third‑generation backend system, detailing the orthogonal decomposition model, three‑tier horizontal layering, key middleware such as MCQ, Motan RPC, SSDCache, and the WatchMan tracing platform that together enable high‑availability, massive concurrency, and low‑latency services for billions of users.
This article presents Sogou's design for a highly available, high‑performance, and horizontally scalable business platform, illustrating key architectural components and scalability strategies through a series of diagrams.
This article compiles recent high‑availability architecture discussions covering plugin‑based system design, simple kanban tools, independent scheduling services, challenges of managing 60‑billion‑row tables, and the trade‑offs between UUIDs and custom distributed ID generators, offering practical insights for backend engineers.
