This article examines the key differences between Commvault and Veritas NetBackup, evaluating their features, cloud integration, endpoint support, cost, and suitability for small versus large enterprises to help organizations choose the most appropriate backup and disaster recovery solution.
The article analyzes the recent Weimob data‑deletion incident, explaining why recovery is complex, comparing on‑premise, hybrid, and full‑cloud database architectures, and outlining the technical steps and obstacles involved in restoring massive lost data.
In this interview, a senior database expert from Huaxia Bank shares twelve years of experience and explains how moving from traditional replication to multi‑active, real‑time consistent data centers, combined with automation and mobile remote operations, is transforming banking database reliability and security.
High AvailabilityMulti-Active Architecturedatabase operations
JDHBase, JD.com’s large‑scale KV store, powers billions of daily reads and writes across 7,000 nodes, and this article details its multi‑active, cross‑region architecture—including HBase replication fundamentals, Fox Manager routing, automatic failover policies, dynamic replication tuning, and serial replication to ensure strong consistency.
The article explains how Alipay’s Logical Data Center (LDC) architecture, with its RZone, GZone, and CZone unitization, combined with OceanBase’s Paxos‑based consensus, enables massive TPS growth, traffic diversion, and disaster‑recovery while navigating the CAP theorem constraints.
The article describes a full‑stack performance monitoring system for WeChat Mini Programs presented by Niu Tifa, covering Mini Program architecture fundamentals, a monitoring system architecture using a JS SDK, Druid, Elasticsearch, and practical applications like load timing, error handling, fallback strategies, with dashboards and alerts, emphasizing low request volume and non‑intrusive monitoring.
The article explores how Alipay’s financial data is protected through multi‑site data centers, hot and cold backups, and disaster‑recovery mechanisms, explaining why destroying a single server—or even multiple facilities—won’t instantly erase users’ funds, and outlining the lengths required to truly cripple the system.
On December 5th Alipay experienced a brief outage that sent users into panic, prompting a humorous recount of personal losses, meme images, and a reminder of the critical importance of high‑availability architecture and disaster‑recovery planning for large‑scale financial services.
This article explains ByteDance's shift to RocketMQ, detailing the business drivers, technical advantages, proxy layer implementation for microservices, encountered challenges, and the disaster‑recovery strategies adopted to ensure high availability and performance.
To mitigate the inherent single‑point‑failure risk of the open‑source InfluxDB community edition, the article proposes deploying multiple InfluxDB instances with concurrent client writes, tracking failed writes, temporarily storing them, and using custom workers to replay data, while addressing timeout, data consistency, and storage considerations.
The article examines how Alipay’s financial data would survive a physical destruction of its servers by explaining multi‑site data center architectures, hot and cold backups, power redundancy, fire‑suppression systems, and the role of partner banks in data recovery, highlighting the extensive resilience measures in modern financial infrastructures.
The article explains how financial platforms like Alipay protect user funds through multi‑site data centers, hot and cold backups, power redundancy, fire‑suppression systems, and strict location standards, showing why destroying a single server would not erase all stored money.
The article explains how Alipay’s financial system uses multi‑site, multi‑center architectures, hot‑standby, active‑active, and cold‑backup strategies, along with stringent A‑class data‑center standards, to ensure that even catastrophic physical attacks cannot erase users' money.
This article summarizes Shen Hui’s presentation on how ByteDance adopted RocketMQ in a massive micro‑service environment, detailing the business background, reasons for choosing RocketMQ, the proxy‑based deployment, encountered challenges, and the multi‑data‑center disaster‑recovery solutions implemented.
This guide explains CloudEndure’s features and walks through a step‑by‑step example of configuring AWS cross‑region disaster recovery, covering initial project setup, data replication, test and recovery mode switching, and the failback process, while highlighting networking and security considerations.
This article reviews Alibaba HBase's evolution toward high availability, covering large‑cluster architecture, reliability metrics (MTTF/MTTR), disaster‑recovery strategies such as data replication and traffic switching, performance optimizations for extreme latency requirements, and lessons learned for building resilient distributed database services.
This article explains the definitions of disaster tolerance, fault tolerance, and disaster recovery, compares their purposes, discusses backup versus disaster‑tolerance solutions, outlines key metrics such as RTO and RPO, and presents common architectural and investment considerations for building resilient enterprise systems.
Effective MySQL backup and disaster recovery are essential for protecting critical business data; this guide explains backup types, tools like Percona XtraBackup, scheduling, local and remote strategies, incremental processes, preparation and restoration steps, and introduces a platform for automated backup management.
This article introduces the global data replication market, explains synchronous and asynchronous replication, and details five typical replication techniques—host‑based, application/middleware‑based, database‑based, storage‑gateway‑based, and storage‑media‑based—highlighting their principles, advantages, and trade‑offs for disaster‑recovery planning.
The article explains how to select and design disaster‑recovery communication links by evaluating distance, bandwidth, transmission media, and multiplexing techniques such as DWDM, FDM, and TDM, while balancing cost, reliability, and application requirements.
The article explains how anticipating and engineering for diverse failure scenarios—from hardware faults and software bugs to traffic spikes and external attacks—can dramatically improve system reliability, reduce downtime, and protect business continuity in modern distributed and cloud environments.
Tencent Cloud Elasticsearch now supports cross‑availability‑zone deployment, requiring even‑numbered data nodes, dedicated master nodes, and replica settings to ensure continuous service when a zone fails, with detailed steps for quick setup and region limitations explained.
The article examines how the rise of financial technology and dual‑mode IT is reshaping banking disaster‑recovery strategies, comparing active‑active and two‑site‑three‑center architectures, and proposing an integrated solution to improve resource utilization, reduce risk, and ensure continuous service.
This article explains how a bank can create a specialized Operations SWAT team, define its role, adopt seven essential “weapons” such as layered monitoring, intelligent alerts, communication protocols, automation, and disaster‑recovery tactics, and continuously train the team to meet strict five‑minute recovery targets.
This article explains how a bank can create a specialized SWAT‑style operations team, define its roles, adopt seven essential "weapons" such as monitoring and intelligent alerts, and apply ten tactical processes—from communication to automation—to meet strict five‑minute recovery and regulatory requirements.
Modern cloud outages reveal that cold‑standby or simple multi‑cloud promises often provide only psychological comfort; achieving true high availability requires active‑active designs with local traffic handling, data partitioning, and low‑latency synchronization, while balancing cost, complexity, and physical distance constraints.
The article explains how Xianyu guarantees high‑availability of its core Ha3‑based search engine through independent gateway deployment, multi‑datacenter disaster recovery, traffic isolation, comprehensive monitoring, pressure testing, gray releases, and automated/manual failover, enabling rapid issue detection, recovery, and continuous service stability.
This article explains Redis’s key features and common use cases, highlights the risks of single‑node deployments, compares two master‑slave replication architectures, and outlines their pros and cons, and details how Redis Sentinel provides automated monitoring, failover, and configuration to achieve high availability.
This article chronicles Ctrip's database high‑availability and disaster‑recovery evolution—from simple SQL Server mirroring in the early years, through SAN‑based clustering and AlwaysOn, to the adoption of MySQL, Redis, MHA, and a one‑click DR automation tool—highlighting architectural decisions, challenges, and operational lessons learned.
This article details how Dell EMC’s SC series storage, VPLEX and RecoverPoint were tested by E Enterprise Research Institute across multiple failure scenarios, demonstrating automatic failover, rapid point‑in‑time restores, flexible backup granularity and millisecond‑level recovery to ensure uninterrupted business operations.
Amid rising cloud outages and strict financial regulations, this article examines traditional multi‑active database solutions such as Oracle RAC and IBM GDPS, contrasts them with modern distributed database designs, and details SequoiaDB’s multi‑active architecture and concrete disaster‑recovery procedures for single‑node, site‑wide, and network failures.
This article reviews traditional database high‑availability techniques, explains the advantages of distributed multi‑replica consistency (Paxos/Raft) used by OceanBase, and compares various deployment topologies—from single‑site three‑replica to multi‑city five‑replica designs—highlighting their trade‑offs and best‑practice recommendations.
This article provides a detailed technical guide on selecting and designing disaster‑recovery communication links, covering distance and bandwidth requirements, fiber types, multiplexing methods, and cost‑effective solutions such as DWDM, SDH, and Ethernet‑based connections.
Meituan Waimai’s mobile client employs a high‑availability architecture built on loosely‑coupled teams, comprehensive monitoring, encrypted logging, multi‑layer disaster recovery, gray‑release strategies, and an incident‑response workflow, enabling rapid detection and resolution of failures while supporting 20 million daily orders.
This article examines Alipay's evolution from a simple single‑datacenter setup to a multi‑active‑active, unit‑based architecture, detailing the technical challenges of high availability, disaster recovery, failover design, blue‑green deployment, and how these solutions enable continuous service during massive traffic spikes like Double 11.
This article examines the importance of high‑availability and disaster‑recovery architectures, tracing Alipay’s evolution from a simple load‑balanced setup through multi‑datacenter, failover, and unit‑based designs that address scalability, data consistency, and continuous service delivery challenges.
This article explains how the seqsvr service in WeChat evolved its disaster‑recovery architecture from a simple master‑slave model to a dynamic routing solution, detailing design principles, lease mechanisms, and operational optimizations that ensure monotonic UID sequences and high availability.
This article provides a comprehensive overview of disaster recovery and business continuity planning, covering essential concepts, DRI best‑practice steps, risk and business impact analysis, design options, implementation phases, and practical exercise methods to ensure resilient IT operations.
DRI best practicesIT Operationsbusiness continuity
The article examines how Continuous Data Protection (CDP) transforms traditional backup and disaster‑recovery approaches by offering near‑real‑time recovery points, finer granularity, and flexible implementation models across applications, files, and storage blocks, while comparing its benefits and trade‑offs with conventional methods.
This article explains how Dada's order platform was refactored from a Python Flask service to a Java‑based microservice architecture, employing load testing, circuit breaking with Hystrix, and multi‑datacenter disaster recovery to reliably handle eight million orders in a single day during the Double‑11 shopping festival.
This article provides a detailed comparison of Oracle RAC and MySQL MHA high‑availability solutions, covering architecture overviews, network requirements, failover methods, configuration examples, disaster‑recovery options, performance trade‑offs, and practical considerations for choosing the right approach.
The article explains how JD’s senior architect prepares for the 11.11 shopping festival by defining high‑availability goals, discussing scalability strategies, disaster‑recovery planning, performance optimization, and system resilience to ensure reliable service under massive traffic spikes.
Alibaba's Global Operations Center (GOC) was honored with the DRI International "Best BCM Innovation Practice" award for its industry‑leading, end‑to‑end business continuity solution that ensures 99.99% service availability through proactive fault prevention, rapid detection, automated recovery, and thorough post‑incident analysis.
Alibaba’s Global Operations Center (GOC) showcases a comprehensive business continuity solution that combines fault prevention, AI‑driven detection, rapid recovery, and automated post‑mortem processes, enabling 99.99% service availability and earning the DRI International Annual Best BCM Innovation Practice Award.
The article describes Qunar's 99.99% high‑availability goal for MySQL databases, outlines its HA designs (MMM, PXC, QMHA), compares hot and cold backup methods, discusses common backup challenges, and details Qunar's comprehensive backup and recovery solution including Percona XtraBackup, dedicated backup nodes, centralized management, and consistency verification.
This article shares practical DevOps tactics—including disaster‑recovery drills, SET architecture, automated self‑healing, and disciplined change management—to help operations teams reduce errors, improve reliability, and free time for strategic work.
This article explains Alibaba Cloud's Server Load Balancer (SLB) architecture and its four-tier high‑availability design—application processing, cluster forwarding, cross‑zone disaster recovery, and cross‑region disaster recovery—detailing both product features and user‑side best practices.
This article compiles 22 practical recommendations from an operations manager covering domain management, CDN usage, image servers, data center selection, monitoring, security, redundancy, high‑availability architecture, disaster‑recovery planning, and team coordination to help ensure stable and secure online services.
This talk shares practical high‑availability designs learned from Alibaba's Taobao platform and Alibaba Cloud, covering traditional IDC stability mechanisms, modern cloud‑native fault‑tolerance, caching tricks, performance tuning, limit‑and‑degrade tactics, disaster‑recovery planning, and multi‑region deployment for handling billions of requests during peak events.
In this talk, Alibaba expert Mu Jian shares how the massive Taobao e‑commerce platform achieved high availability through layered networking, cache design, OS‑level tuning, rate limiting, disaster‑recovery planning, and cloud‑native architectures, offering practical guidance for building resilient systems at scale.
This article reviews core database high‑availability technologies—including IBM DB2 HADR, HACMP clustering, SQL and Q replication, storage‑level disaster recovery, and DPF considerations—explaining their mechanisms, suitable scenarios, and practical deployment tips for enterprise environments.
This article introduces XPipe, a framework designed by Ctrip to enable Redis multi‑data‑center replication, ensure high availability through keeper and MetaServer components, and provide reliable disaster‑recovery (DR) switching while maintaining low latency and data consistency.
This article presents a detailed walkthrough of a typical web system architecture, covering layered components, load‑balancing strategies, business service and communication layers, storage options from file to object stores, and key criteria for evaluating architecture such as cost, scalability, security, and disaster recovery.
This article explains a comprehensive database backup workflow—including daily full backups with xtrabackup, real‑time binlog incremental backups, multi‑level retention policies, automated failure detection, disaster‑recovery across multiple data centers, and fast table‑level restoration—to help prevent prolonged outages.
An in‑depth look at Blizzard’s Hearthstone outage reveals that a power interruption likely triggered storage failures in an Oracle RAC cluster, leading to corrupted blocks, incomplete recovery, and a reliance on backups, while highlighting broader lessons on database backup, disaster recovery, and operational decision‑making.
The article details Ctrip's call‑center architecture evolution, describing the multi‑layer active‑active design, public access, application and client layers, unified login mechanisms, operational challenges, disaster‑recovery drills, and future plans for software‑only and mobile agents, illustrating practical SRE principles in a large‑scale telephony system.
This article traces the history of disaster recovery from its 1970s origins through the rise of hot‑standby sites, the impact of internet expansion, and the shift to cloud‑based DRaaS, while forecasting future trends such as integrated business‑as‑disaster solutions, automated management platforms, and social‑media‑driven emergency communication.
This article explains how to design and implement a reliable etcd disaster‑recovery system for a Kubernetes‑based platform, covering backup scheduling, data synchronization across sites, code structure, and the advantages and drawbacks of the approach.
This article explains VMware Site Recovery Manager's architecture, its hybrid and private cloud disaster‑recovery approaches using vSphere Replication and storage‑array replication, compares it with Microsoft Azure Site Recovery, and outlines the use cases for vSAN Stretched Cluster solutions.
This article explains Microsoft Azure Site Recovery’s hybrid and private‑cloud disaster‑recovery solutions, covering basic concepts, architecture, configuration steps, integration with SCVMM, and recent feature enhancements for continuous VM replication, failover testing, and customizable recovery workflows.
This article details Shandong Mobile's cloud‑based dual‑active architecture, covering virtualization, VMware HA/DRS, Oracle GoldenGate and ADG optimizations, large Layer‑2 network designs, heartbeat mechanisms to avoid split‑brain, comprehensive testing scenarios, and the resulting reductions in downtime and hardware costs.
This article examines MySQL's semi‑synchronous replication, analyzes consistency scenarios across master‑slave failures and switches, and highlights the difficulties of data rollback and multi‑master conflicts that affect high‑availability clusters.
Data ConsistencyHigh Availabilitydisaster recovery
This article details how Tencent's operations team orchestrated a seamless, zero‑impact migration of over 70 million users across three data centers during the 2015 Tianjin explosion, highlighting the four key capabilities—distribution, scheduling, data synchronization, and automated operations—that enabled multi‑active disaster recovery at massive scale.
The article explains how modern call centers evolve to multi‑site active‑active deployments, detailing the network design, disaster‑recovery mechanisms, login and monitoring logic, key features supporting automatic failover for thousands of agents, and future expansion possibilities.
The article describes how a large‑scale call center evolves to a multi‑site architecture and implements system‑level active‑active disaster recovery using Ctrip's contact‑center and unified login platforms, detailing the login flow, fault‑detection logic, key features, and future extensions.
The article proposes a cross‑IDC disaster‑recovery architecture for key‑value stores that uses a storage proxy layer to record write logs in the primary data center, forward them to a read‑only backup center for replay, and route reads via load‑balancing, while noting current limitations such as complex configuration, low synchronization concurrency, and lack of multi‑write support.
Facing rapid growth and regulatory pressure in 2015, a leading Chinese internet finance firm partnered with Shanghai Aikesheng to design a multi‑stage, MySQL‑based cloud database solution—starting with a private cloud trial, then disaster‑recovery, distributed clustering for massive red‑packet traffic, and finally evolving into an enterprise‑grade PaaS offering.
This comprehensive guide explains how SQL Server log shipping works, details the roles of primary, secondary, and monitor servers, walks through each job type, discusses execution intervals and data‑loss implications, and provides step‑by‑step failover and troubleshooting procedures.
This article chronicles how WeChat’s backend evolved from a simple messaging prototype to a globally distributed, multi‑data‑center system, detailing its message model, unified sync protocol, three‑layer architecture, platformization, disaster‑recovery design, performance tuning, and emerging resource‑scheduling challenges.
This article explains how large‑scale elastic architecture designs achieve high availability and scalability by automatically allocating resources during traffic spikes such as Spring Festival ticket sales and Double‑11 shopping festivals, and outlines key techniques, trends, and disaster‑recovery strategies.
The article examines system reliability concepts such as availability, MTBF, MTTR, and outage classifications, analyzes the Alipay service interruption, discusses various redundancy and failover strategies, and explores YunOS reliability testing and design practices to improve overall system robustness.
When a massive container explosion threatened Tencent's Tianjin data center, the operations team executed a 24‑hour, continent‑wide user migration that moved over 200 million QQ users to Shenzhen and Shanghai without service interruption, showcasing unprecedented disaster‑recovery capabilities.
This article summarizes Chen Ming’s 2015 ArchSummit talk on WeChat Moments, detailing its massive posting volume, core database tables, publishing and reading workflows, and the multi‑data‑center disaster‑recovery architecture that ensures seamless, loss‑less service worldwide.
This article explains why protecting database‑stored sensitive data is critical, analyzes common attack vectors and breach statistics, and provides detailed logical and physical security measures—including user and permission management, auditing, encryption, disaster‑recovery planning, and Oracle replication technologies—to help organizations achieve comprehensive database protection.
This article shares practical insights from a seasoned KVM specialist on how to monitor hardware, set up alerts, design disaster‑recovery strategies, choose optimal software and hardware, and evaluate public‑cloud providers when migrating workloads to a virtualized environment.
This article explains XenServer's high‑performance virtualization features, OVF/OVA export capabilities, snapshot‑based backup methods, disaster‑recovery replication, XenCenter management console functions, integration with Microsoft System Center, and the open XenAPI that enables third‑party automation.
This article explains how to objectively assess cloud service reliability by examining three key aspects—availability, access control, and disaster recovery—and provides practical strategies such as redundancy design, gradual deployment, automation, and robust backup to improve overall cloud service trustworthiness.
The article explains why high‑traffic web pages need robust fallback and disaster‑recovery mechanisms, outlines common failure scenarios, and presents practical solutions such as request retries, local caching, backup interfaces, and hard‑coded data with illustrative JavaScript code.
This interview with Alibaba's Lin Hao (aka Bi Xuan) explains why the company pursued a multi‑active, geographically distributed data‑center architecture, how it was deployed for the 2014 Double‑11 event, and the technical challenges—including latency, routing consistency, and data synchronization—that had to be solved to achieve high availability and low‑cost disaster recovery.
In this interview, senior database engineer Eric shares the three‑layer J2EE architecture, Oracle and WebLogic stack, security measures, high‑concurrency handling, disaster‑recovery practices, and performance‑tuning techniques used in China’s social security systems.
This guide introduces Linux Logical Volume Manager (LVM), explains its core concepts such as physical volumes, volume groups, and logical volumes, and provides step‑by‑step procedures for checking versions, handling common failures, and performing disaster‑recovery operations on both server and desktop systems.