BestHub
Sign in
Tagged articles

Fault Tolerance

328 articles · Page 3 of 4
Architecture Digest
Architecture Digest
Dec 9, 2020 · Backend Development

Implementing Distributed Locks with Redis: Concepts, Algorithms, and Code Examples

This article explains how to implement distributed locks using Redis, covering the essential requirements of mutual exclusion, deadlock avoidance, and fault tolerance, detailing single‑instance and multi‑instance algorithms, code examples with SETNX and Lua scripts, and discussing challenges such as latency, crashes, and persistence.

Fault Tolerancebackend developmentconcurrency
0 likes · 10 min read
Implementing Distributed Locks with Redis: Concepts, Algorithms, and Code Examples
Manbang Technology Team
Manbang Technology Team
Nov 23, 2020 · Operations

Designing a Comprehensive Stability Assurance System for Large‑Scale Internet Services at Manbang

This article explains how Manbang built a rigorous stability‑assurance framework—including strict fault grading, a "watch‑and‑protect" system, blue‑green deployments, online pressure testing, fault‑drill platforms, and runtime metadata—to ensure rapid iteration while maintaining high availability for millions of logistics users.

Fault Tolerance
0 likes · 12 min read
Designing a Comprehensive Stability Assurance System for Large‑Scale Internet Services at Manbang
Tencent Cloud Developer
Tencent Cloud Developer
Nov 19, 2020 · Backend Development

Kafka Message Queue Reliability Design and Implementation

The article thoroughly explains Kafka’s message‑queue reliability design and implementation, covering use‑case scenarios, core concepts, storage format, producer acknowledgment settings, broker replication mechanisms (ISR, HW, LEO), consumer delivery semantics, the epoch solution for synchronization, and practical configuration guidelines for various consistency and availability requirements.

BrokerFault ToleranceKafka
0 likes · 15 min read
Kafka Message Queue Reliability Design and Implementation
JavaEdge
JavaEdge
Oct 24, 2020 · Databases

Mastering Redis Cluster: Scaling, Routing, and Fault Tolerance Explained

This article explains why Redis clusters are needed, how CLUSTER MEET builds the network, slot assignment, scaling procedures, client redirection mechanisms, batch operations, fault detection, recovery processes, and common operational pitfalls, providing practical guidance for building and maintaining a robust Redis Cluster deployment.

ClusterFault ToleranceRedis
0 likes · 22 min read
Mastering Redis Cluster: Scaling, Routing, and Fault Tolerance Explained
DevOps
DevOps
Oct 20, 2020 · Cloud Computing

Chaos Monkey and the Simian Army: Building Resilient Cloud Systems

The article explains how Netflix uses Chaos Monkey and a suite of related tools, collectively called the Simian Army, to deliberately inject failures into their cloud infrastructure, continuously test fault‑tolerance, and ensure high availability and reliability for their streaming service.

Chaos EngineeringFault ToleranceNetflix
0 likes · 7 min read
Chaos Monkey and the Simian Army: Building Resilient Cloud Systems
IT Architects Alliance
IT Architects Alliance
Oct 13, 2020 · Cloud Native

Designing Fault‑Tolerant Microservices Architecture

Microservice architectures increase system complexity and failure rates, so this article explains key reliability patterns—such as graceful degradation, change management, health checks, self‑healing, fallback caches, retry logic, rate limiting, circuit breakers, and testing—to help engineers design resilient, high‑availability services.

Cloud NativeFault ToleranceMicroservices
0 likes · 23 min read
Designing Fault‑Tolerant Microservices Architecture
Architects' Tech Alliance
Architects' Tech Alliance
Oct 12, 2020 · Operations

Designing Resilient Microservices: Patterns for Fault Tolerance and Failure Management

This article examines the inherent risks of microservice architectures and presents practical patterns—such as graceful degradation, change management, health checks, self‑healing, fallback caching, retries, rate limiting, bulkheads, and circuit breakers—to build highly available, fault‑tolerant services.

BulkheadFault ToleranceMicroservices
0 likes · 15 min read
Designing Resilient Microservices: Patterns for Fault Tolerance and Failure Management
Top Architect
Top Architect
Oct 11, 2020 · Cloud Native

Using Hystrix for Fault Tolerance in Spring Cloud Microservices

This article explains how to integrate Netflix Hystrix into Spring Cloud applications to provide request timeout, circuit‑breaker, fallback, monitoring and resource isolation for microservice calls, including Maven setup, annotation usage, Feign client fallback configuration and disabling options.

Fault ToleranceFeignHystrix
0 likes · 9 min read
Using Hystrix for Fault Tolerance in Spring Cloud Microservices
Architect
Architect
Oct 6, 2020 · Backend Development

Implementing Hystrix for Fault Tolerance in Spring Cloud Microservices

This article explains why microservice calls need fault‑tolerance mechanisms, introduces Hystrix’s core features such as timeouts, circuit‑breaker, fallback, monitoring and resource isolation, and provides step‑by‑step code examples for integrating Hystrix and Feign in a Spring Cloud project.

Fault ToleranceFeignHystrix
0 likes · 8 min read
Implementing Hystrix for Fault Tolerance in Spring Cloud Microservices
DataFunTalk
DataFunTalk
Oct 2, 2020 · Big Data

Single-Task Recovery in Flink: Design and Implementation for Real‑Time Stream Processing

This article describes ByteDance's single‑task recovery solution for Flink's real‑time computation, detailing the problem of global job restarts, the proposed network‑layer enhancements, upstream and downstream optimizations, JobManager restart strategy, implementation challenges, and the measurable latency and availability benefits achieved in production.

Fault ToleranceFlinkSingle-Task Recovery
0 likes · 11 min read
Single-Task Recovery in Flink: Design and Implementation for Real‑Time Stream Processing
Xianyu Technology
Xianyu Technology
Sep 27, 2020 · Backend Development

Design of an Asynchronous Component with Monitoring, Fault Tolerance, and Zero‑Cost Integration

The article presents a design for an asynchronous component that is monitorable, fault‑tolerant, and integrates with zero overhead, compares Akka, RxJava, and a custom JUC‑based implementation, and selects the latter—using extended Callables and a CountDownLatch—to track business units, handle timeouts, and provide fallback behavior.

AsynchronousFault ToleranceJUC
0 likes · 8 min read
Design of an Asynchronous Component with Monitoring, Fault Tolerance, and Zero‑Cost Integration
Architect's Tech Stack
Architect's Tech Stack
Sep 21, 2020 · Backend Development

Overview of Tars: A High‑Performance RPC Framework and Service Governance Platform

The article introduces Tars, an open‑source high‑performance RPC framework and integrated service governance platform derived from Tencent's internal microservice architecture, detailing its design philosophy, layered architecture, core features such as the Tars protocol, load balancing, fault and overload protection, and centralized configuration management.

Fault ToleranceLoad BalancingMicroservices
0 likes · 11 min read
Overview of Tars: A High‑Performance RPC Framework and Service Governance Platform
21CTO
21CTO
Sep 12, 2020 · Fundamentals

Why Distributed Systems Mirror Single‑Node Concurrency and How to Avoid Common Pitfalls

This article explains how concurrency issues that appear in single‑threaded programs become amplified in distributed systems, covering consistency models, network reliability, clock synchronization, fault detection, backpressure, and cascading failures, and offers practical design and testing strategies to build resilient architectures.

Fault Toleranceconcurrencyconsistency
0 likes · 19 min read
Why Distributed Systems Mirror Single‑Node Concurrency and How to Avoid Common Pitfalls
Sohu Tech Products
Sohu Tech Products
Sep 2, 2020 · Backend Development

Implementing Distributed Locks with Redis and Redisson: Abstraction, Auto-Release, and Fault Tolerance

This article explains how to use Redis and Redisson for distributed locking, introduces an abstract DistributedLock interface for flexible implementations, demonstrates automatic lock release with functional callbacks, and discusses fallback strategies and monitoring to ensure reliability in backend systems.

Fault ToleranceRedisRedisson
0 likes · 5 min read
Implementing Distributed Locks with Redis and Redisson: Abstraction, Auto-Release, and Fault Tolerance
Programmer DD
Programmer DD
Jul 31, 2020 · Fundamentals

What Is Distributed Architecture and Why It Powers Modern Systems

This article explains the concept of distributed architecture, its evolution from monolithic systems, core design principles, common challenges such as network latency and data consistency, and how major tech companies adopt it to achieve high availability, scalability, and fault tolerance.

Fault ToleranceSystem Design
0 likes · 11 min read
What Is Distributed Architecture and Why It Powers Modern Systems
Programmer DD
Programmer DD
Jul 28, 2020 · Fundamentals

What Makes Distributed Architecture Essential for Modern Systems?

Distributed architecture, built on distributed computing technologies like J2EE, transforms monolithic systems into multi‑layer, fault‑tolerant platforms by decoupling services, ensuring high availability, scalability, and resilience, while addressing challenges such as network latency, data consistency, and system complexity, as illustrated by real‑world case studies.

Case studyFault ToleranceSystem Design
0 likes · 11 min read
What Makes Distributed Architecture Essential for Modern Systems?
Qunar Tech Salon
Qunar Tech Salon
Jun 16, 2020 · Operations

Qunar's Multi-IDC Deployment and Fault Self‑Healing Architecture

This article describes how Qunar scaled its IDC infrastructure, introduced multi‑IDC deployment, automated DNS‑based load balancing, open‑source DNSDB, and an IDC proxy built on Squid to achieve rapid fault self‑healing and transparent traffic switching for both user and third‑party access.

DNSFault ToleranceLoad Balancing
0 likes · 8 min read
Qunar's Multi-IDC Deployment and Fault Self‑Healing Architecture
Top Architect
Top Architect
Jun 10, 2020 · Fundamentals

A Comprehensive Guide to Learning Distributed Systems

This article provides a thorough overview of distributed systems, explaining their definition, core challenges, key characteristics, essential components, common protocols, and practical implementations to help readers build a solid, structured learning path for mastering distributed architectures.

Fault ToleranceSystem Designdistributed systems
0 likes · 16 min read
A Comprehensive Guide to Learning Distributed Systems
58 Tech
58 Tech
May 22, 2020 · Backend Development

Design and Implementation of a Distributed Retry System Based on Distributed Scheduling

This article presents a comprehensive distributed retry system that leverages a distributed scheduling mechanism to ensure eventual consistency, reduce manual recovery costs, and provide flexible retry strategies, automatic recovery detection, visual management, rate limiting, and intelligent retry for backend services.

Fault ToleranceRetry Mechanismbackend development
0 likes · 13 min read
Design and Implementation of a Distributed Retry System Based on Distributed Scheduling
Big Data Technology & Architecture
Big Data Technology & Architecture
Apr 13, 2020 · Fundamentals

Understanding Replication, Consistency, Fault Tolerance, and the CAP Theorem in Distributed Systems

This article explains the core concepts of replication, consistency, and fault tolerance in distributed systems, discusses strong and asynchronous replication methods, and details the CAP theorem with its consistency, availability, and partition tolerance trade‑offs, illustrating AP and CP scenarios such as Eureka and Zookeeper clusters.

CAP theoremFault Toleranceconsistency
0 likes · 7 min read
Understanding Replication, Consistency, Fault Tolerance, and the CAP Theorem in Distributed Systems
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Mar 23, 2020 · Fundamentals

15 Timeless Architecture Principles Every Engineer Should Follow

This article outlines how to create solid software architectures by presenting a process for forming design principles, detailing fifteen universal architecture guidelines, and explaining service‑splitting and key design rules that together help build scalable, maintainable, and resilient systems.

Design PrinciplesFault ToleranceMicroservices
0 likes · 16 min read
15 Timeless Architecture Principles Every Engineer Should Follow
Architecture Digest
Architecture Digest
Mar 11, 2020 · Big Data

Apache Flink: Unified Stream and Batch Processing Architecture and Core Concepts

This article provides a comprehensive overview of Apache Flink, explaining how it unifies stream and batch processing on a single runtime, detailing its key features, APIs, libraries, architectural components, fault‑tolerance mechanisms, scheduling, iterative processing, and back‑pressure monitoring.

Apache FlinkBatch ProcessingDistributed Computing
0 likes · 20 min read
Apache Flink: Unified Stream and Batch Processing Architecture and Core Concepts
360 Tech Engineering
360 Tech Engineering
Mar 10, 2020 · Fundamentals

Introduction to Raft: A Comprehensive Overview of the Distributed Consensus Algorithm

This article provides a thorough introduction to the Raft consensus algorithm, explaining its purpose, core components such as state machine replication, log and consensus module, leader‑follower model, client interaction, fault‑tolerance considerations, the CAP trade‑off, and why Go is a suitable implementation language.

Fault ToleranceGoRaft
0 likes · 11 min read
Introduction to Raft: A Comprehensive Overview of the Distributed Consensus Algorithm
Youzan Coder
Youzan Coder
Feb 28, 2020 · Big Data

Flink Checkpoint Principle Analysis and Failure Cause Investigation

The article thoroughly explains Apache Flink’s checkpoint mechanism—including state types, coordinator workflow, exactly‑once versus at‑least‑once semantics, common failure sources such as code exceptions, storage or network issues, and practical configuration tips like interval settings, local recovery and externalized checkpoints.

Apache FlinkCheckpointFault Tolerance
0 likes · 15 min read
Flink Checkpoint Principle Analysis and Failure Cause Investigation
Architects' Tech Alliance
Architects' Tech Alliance
Feb 10, 2020 · Fundamentals

Mastering Distributed System Fundamentals: Models, Replication, Consistency, and Protocols

This article provides a comprehensive overview of distributed system fundamentals, covering node modeling, replica concepts, consistency levels, data distribution strategies, centralized and decentralized replica protocols, lease mechanisms, quorum, two‑phase commit, MVCC, Paxos, and the CAP theorem, while analyzing their trade‑offs in availability, consistency, and partition tolerance.

Consensus ProtocolsData DistributionFault Tolerance
0 likes · 55 min read
Mastering Distributed System Fundamentals: Models, Replication, Consistency, and Protocols
Architects' Tech Alliance
Architects' Tech Alliance
Feb 4, 2020 · Backend Development

Microservice Architecture Evolution: From Monolith to Service Mesh

This article walks through the transformation of an online supermarket from a simple monolithic website to a fully fledged microservice architecture, highlighting the motivations, design decisions, common pitfalls, and essential components such as monitoring, tracing, logging, gateways, service discovery, circuit breaking, testing strategies, and service mesh adoption.

Fault ToleranceMicroservicesMonitoring
0 likes · 22 min read
Microservice Architecture Evolution: From Monolith to Service Mesh
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Jan 7, 2020 · Backend Development

Spring Cloud vs Dubbo: Protocol Handling, Performance, Load Balancing, Fault Tolerance, and Routing in Microservice Architecture

This article compares Spring Cloud and Dubbo across protocol handling, performance tuning, load‑balancing strategies, fault‑tolerance mechanisms, and routing/traffic‑shaping features, highlighting their flexibility, configuration complexity, and suitability for different microservice scenarios.

DubboFault ToleranceLoad Balancing
0 likes · 6 min read
Spring Cloud vs Dubbo: Protocol Handling, Performance, Load Balancing, Fault Tolerance, and Routing in Microservice Architecture
Efficient Ops
Efficient Ops
Dec 17, 2019 · Operations

How Alibaba Scales Flink: Lessons in Big Data Operations

This article details Alibaba's massive Flink deployment, covering its historical background, the operational challenges of managing tens of thousands of nodes, the design of a comprehensive Flink management platform, and the automated solutions for fault handling, resource allocation, and performance testing in a large‑scale big‑data environment.

AutomationBig Data OperationsCluster Management
0 likes · 20 min read
How Alibaba Scales Flink: Lessons in Big Data Operations
21CTO
21CTO
Nov 15, 2019 · Operations

How SRE Designs Highly Available Software Systems at Scale

This article presents Google SRE expert Ramón Medrano Llamas’s comprehensive guide on designing, operating, and maintaining large‑scale, highly available software systems, covering SRE fundamentals, daily workflows, scalability strategies, fault‑tolerant architecture, monitoring, and operational best practices.

Fault ToleranceMonitoringSRE
0 likes · 13 min read
How SRE Designs Highly Available Software Systems at Scale
Java High-Performance Architecture
Java High-Performance Architecture
Nov 12, 2019 · Backend Development

How Kafka Consumer Groups Boost Performance and Fault Tolerance

Kafka consumer groups enable multiple consumers to share partition workloads, ensuring exclusive consumption within a group, flexible consumption patterns like broadcast and unicast, and automatic fault‑tolerance through rebalancing, ultimately improving throughput, scalability, and resilience of streaming applications.

Fault ToleranceKafkabackend development
0 likes · 4 min read
How Kafka Consumer Groups Boost Performance and Fault Tolerance
Architecture Digest
Architecture Digest
Nov 9, 2019 · Backend Development

Design and Implementation of eBay's Next‑Generation Million‑TPS Core Accounting System

The article details eBay's 2018‑2020 design, performance testing, and fault‑tolerance architecture of a next‑generation core accounting system capable of handling millions of transactions per second, covering system goals, multi‑region deployment, event‑sourcing, Raft consensus, scalability optimizations, and the planned open‑source release.

Distributed ArchitectureFault ToleranceHigh TPS
0 likes · 24 min read
Design and Implementation of eBay's Next‑Generation Million‑TPS Core Accounting System
Architects' Tech Alliance
Architects' Tech Alliance
Oct 2, 2019 · Operations

Understanding Disaster Tolerance, Fault Tolerance, and Disaster Recovery: Concepts, Differences, and Implementation Strategies

This article explains the definitions of disaster tolerance, fault tolerance, and disaster recovery, compares their purposes, discusses backup versus disaster‑tolerance solutions, outlines key metrics such as RTO and RPO, and presents common architectural and investment considerations for building resilient enterprise systems.

BackupDisaster RecoveryFault Tolerance
0 likes · 8 min read
Understanding Disaster Tolerance, Fault Tolerance, and Disaster Recovery: Concepts, Differences, and Implementation Strategies
Alibaba Cloud Native
Alibaba Cloud Native
Sep 25, 2019 · Cloud Native

Mastering Distributed System Design: Patterns, Performance, and Fault Tolerance

This article provides a comprehensive overview of distributed system architecture, covering essential design patterns such as gateways, sidecars, and service meshes, performance techniques like caching and async communication, fault‑tolerance mechanisms including rate limiting and circuit breakers, and practical DevOps practices for deployment and monitoring.

Cloud NativeFault ToleranceGateway
0 likes · 13 min read
Mastering Distributed System Design: Patterns, Performance, and Fault Tolerance
Architecture Digest
Architecture Digest
Sep 23, 2019 · Operations

Improving Application Availability: Practices, Monitoring, and Fault‑Tolerance in a Large‑Scale Payment System

The article describes how a high‑traffic payment platform achieves 99.999% availability by avoiding single points of failure, applying fail‑fast principles, implementing resource limits, building real‑time monitoring and alerting, and automating fault detection, routing, and recovery to ensure continuous 7×24 operation.

Fault ToleranceHigh AvailabilityMonitoring
0 likes · 23 min read
Improving Application Availability: Practices, Monitoring, and Fault‑Tolerance in a Large‑Scale Payment System
Java Captain
Java Captain
Sep 19, 2019 · Backend Development

A Comprehensive Overview of Microservice Architecture and Its Evolution

This article presents a detailed, step‑by‑step illustration of microservice architecture, covering its motivations, component breakdown, migration from monoliths, common pitfalls, monitoring, tracing, logging, gateway, service discovery, resilience patterns, testing strategies, frameworks, and the emerging service‑mesh approach.

Fault ToleranceMonitoringservice mesh
0 likes · 23 min read
A Comprehensive Overview of Microservice Architecture and Its Evolution
AntTech
AntTech
Sep 11, 2019 · Artificial Intelligence

ElasticDL: An Open‑Source Elastic Deep Learning System Built on TensorFlow 2.0 and Kubernetes

ElasticDL, the first industry‑level open‑source system for elastic deep learning on TensorFlow, leverages Kubernetes‑native scheduling, fault‑tolerance, and TensorFlow 2.0 Eager Execution to dramatically improve cluster utilization, simplify distributed training, and integrate seamlessly with tools like Kubeflow and SQLFlow.

Distributed Deep LearningElasticDLFault Tolerance
0 likes · 13 min read
ElasticDL: An Open‑Source Elastic Deep Learning System Built on TensorFlow 2.0 and Kubernetes
dbaplus Community
dbaplus Community
Sep 10, 2019 · Big Data

Why Exactly‑Once Processing Is So Hard in Distributed Systems (And How to Tackle It)

This article explores the two toughest problems in distributed stream processing—exactly‑once message handling and ordering—by dissecting the underlying impossibility of perfect failure detectors, the liveness‑vs‑safety trade‑off, zombie processes, and the practical solutions employed by systems such as Flink, Kafka Streams, MillWheel, and Spark.

ConsensusFault ToleranceStream Processing
0 likes · 81 min read
Why Exactly‑Once Processing Is So Hard in Distributed Systems (And How to Tackle It)
Big Data Technology & Architecture
Big Data Technology & Architecture
Jun 19, 2019 · Big Data

Understanding Spark Structured Streaming StateStore: Architecture, Operations, and Fault Recovery

This article explains the design and implementation of Spark Structured Streaming's StateStore module, covering its distributed architecture, state sharding, versioning, batch read/write, migration, update/query APIs, maintenance compaction, and fault‑tolerance mechanisms that enable incremental continuous queries with exactly‑once guarantees.

Big DataFault ToleranceSpark
0 likes · 8 min read
Understanding Spark Structured Streaming StateStore: Architecture, Operations, and Fault Recovery
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Jun 5, 2019 · Fundamentals

Understanding Paxos: A Beginner’s 30‑Minute Guide with Real‑World Analogy

This article explains the Paxos consensus algorithm in plain terms, using a relatable travel‑planning analogy to illustrate how proposers, acceptors, and majority voting achieve fault‑tolerant agreement in distributed systems, and connects the concept to real‑world implementations like Google’s Chubby and ZooKeeper.

AlgorithmFault TolerancePaxos
0 likes · 13 min read
Understanding Paxos: A Beginner’s 30‑Minute Guide with Real‑World Analogy
dbaplus Community
dbaplus Community
May 25, 2019 · Backend Development

Mastering Thread‑Pool Isolation: Prevent Cascading Failures in Java Services

This article explains the concept of fault tolerance in software architecture, illustrates why thread‑pool isolation is essential for preventing cascading failures, and provides concrete Java implementations—including code examples, pros and cons, and practical guidance for applying the technique in real‑world backend systems.

Fault ToleranceIsolationThread Pool
0 likes · 10 min read
Mastering Thread‑Pool Isolation: Prevent Cascading Failures in Java Services
21CTO
21CTO
Apr 29, 2019 · Big Data

How EasyScheduler Powers Scalable Big Data Workflow Management

EasyScheduler is an open‑source big‑data workflow scheduler that uses a decentralized architecture with Master and Worker nodes coordinated via ZooKeeper, supporting DAG‑based task definitions, various task types, fault tolerance, priority handling, distributed locks, and remote logging, all illustrated with detailed component diagrams.

Big DataDAGFault Tolerance
0 likes · 17 min read
How EasyScheduler Powers Scalable Big Data Workflow Management
Architecture Digest
Architecture Digest
Apr 29, 2019 · Big Data

EasyScheduler: An Open‑Source Big Data Workflow Scheduling System – Architecture and Design Overview

This article introduces EasyScheduler, an open‑source big data workflow scheduling system, explaining its core terminology, decentralized architecture, distributed lock implementation, thread‑shortage handling, fault‑tolerance mechanisms, task‑retry and priority designs, as well as its logging solution using Logback and gRPC.

DAGFault ToleranceWorkflow
0 likes · 14 min read
EasyScheduler: An Open‑Source Big Data Workflow Scheduling System – Architecture and Design Overview
ITPUB
ITPUB
Mar 26, 2019 · Operations

How to Build a 99.99% High‑Availability Service: Practices and Architecture Evolution

This article explains the essential requirements for achieving 99.99% service availability—consistency, eliminating single points, placement groups, traffic isolation, same‑city active‑active, N+1 redundancy, and multi‑region active‑active—illustrated with a step‑by‑step Yum repository service case study and evolving architecture diagrams.

Cloud OperationsFault Tolerancearchitecture
0 likes · 9 min read
How to Build a 99.99% High‑Availability Service: Practices and Architecture Evolution
iQIYI Technical Product Team
iQIYI Technical Product Team
Mar 15, 2019 · Cloud Computing

Design and Architecture of QLive Large‑Scale Live Streaming Service

The QLive service powers iQIYI’s massive live‑streaming events—such as the Spring Festival Gala—by combining vertical and horizontal scaling, a three‑layer architecture with dual data‑center isolation, multi‑level caching, circuit‑breaker/degradation controls, and a Flume‑Kafka‑Hive monitoring pipeline to sustain over 400 k QPS and 99.9999 % availability.

Fault ToleranceMonitoringVertical Scaling
0 likes · 9 min read
Design and Architecture of QLive Large‑Scale Live Streaming Service
Big Data Technology & Architecture
Big Data Technology & Architecture
Mar 13, 2019 · Big Data

Understanding Fault Tolerance and Exactly-Once Semantics in Apache Flink

This article explains Apache Flink's fault‑tolerance mechanisms, including checkpointing, barrier alignment, the differences between At‑Least‑Once and Exactly‑Once semantics, configuration options, incremental checkpointing, and the requirements for external sources and sinks to achieve end‑to‑end exactly‑once processing.

Apache FlinkBig DataFault Tolerance
0 likes · 15 min read
Understanding Fault Tolerance and Exactly-Once Semantics in Apache Flink
JD Tech
JD Tech
Mar 6, 2019 · Backend Development

Understanding Hystrix: Why It’s Needed and How to Use It for Dependency Isolation

This article explains why Hystrix is essential for large distributed systems, describes its dependency isolation mechanisms—including command, group, thread‑pool, and semaphore isolation—covers circuit‑breaker behavior, fallback strategies, and provides detailed Java code examples for configuration and usage.

Fault ToleranceHystrixMicroservices
0 likes · 12 min read
Understanding Hystrix: Why It’s Needed and How to Use It for Dependency Isolation
Java Architect Essentials
Java Architect Essentials
Feb 25, 2019 · Backend Development

Service Isolation Design: Principles, Methods, and Best Practices

The article explains service isolation in system architecture, its origins, why it matters, two main isolation approaches (by service and by user), their advantages and drawbacks, and key considerations to ensure fault containment and improve overall system availability.

Fault ToleranceMicroservicesbackend design
0 likes · 7 min read
Service Isolation Design: Principles, Methods, and Best Practices
dbaplus Community
dbaplus Community
Feb 18, 2019 · Databases

How Do Fault‑Tolerant Transactions Work? Exploring Raft, KV Engines, and Concurrency Control

This article examines multiple fault‑tolerant transaction designs—RSM‑based KV, RSM‑based transactions, shared‑storage approaches, high‑availability KV layers, and single‑node engine extensions—comparing their replication strategies, lock handling, and performance trade‑offs while raising open questions about ordering and consistency.

Fault ToleranceKV storeRaft
0 likes · 15 min read
How Do Fault‑Tolerant Transactions Work? Exploring Raft, KV Engines, and Concurrency Control
Architects Research Society
Architects Research Society
Jan 19, 2019 · Cloud Native

Three Common Microservices Integration Pitfalls and Their Mitigation Strategies

This article examines three frequent pitfalls encountered when integrating microservices—complex communication, asynchronous challenges, and distributed transaction difficulties—and proposes mitigation techniques such as rapid failure handling, workflow engines, timeout management, and compensation patterns to improve resilience and reduce system complexity.

Cloud NativeFault ToleranceMicroservices
0 likes · 13 min read
Three Common Microservices Integration Pitfalls and Their Mitigation Strategies
Programmer DD
Programmer DD
Dec 21, 2018 · Backend Development

How Circuit Breakers Safeguard Distributed Systems from Cascading Failures

This article explains the concept of circuit breaking in distributed systems, outlines a four‑step implementation process with strategies for detecting unhealthy services, cutting off calls, probing recovery, and restoring normal operation, and shares best‑practice tips to minimize downtime and improve resilience.

Fault Tolerancecircuit breakerdistributed systems
0 likes · 10 min read
How Circuit Breakers Safeguard Distributed Systems from Cascading Failures
Architect's Tech Stack
Architect's Tech Stack
Dec 5, 2018 · Operations

Practical Fault‑Tolerance Practices in a Large‑Scale Activity Operations Platform

The article shares a comprehensive, experience‑driven guide on building fault‑tolerant systems—covering retry mechanisms, dynamic node removal, timeout settings, service degradation, decoupling, and business‑level safeguards—to enable a platform that scales from millions to billions of daily requests without relying on manual fire‑fighting.

Fault ToleranceOperationsRetry Mechanism
0 likes · 21 min read
Practical Fault‑Tolerance Practices in a Large‑Scale Activity Operations Platform
Programmer DD
Programmer DD
Oct 30, 2018 · Fundamentals

What Is Paxos? A Storytelling Guide to Distributed Consensus

This article uses a vivid allegorical story to introduce the Paxos algorithm, then explains its roles, two-phase protocol, fault assumptions, and why majority and multiple acceptors are essential for achieving reliable consensus in distributed systems.

AlgorithmFault TolerancePaxos
0 likes · 10 min read
What Is Paxos? A Storytelling Guide to Distributed Consensus
UC Tech Team
UC Tech Team
Oct 23, 2018 · Operations

Understanding Faults and Fault Isolation Strategies in Distributed Systems

The article explains what constitutes a fault, introduces key metrics such as RPO and RTO, and describes various fault isolation principles, patterns, and practical examples—including dependency degradation, failover, dynamic adjustment, fast‑fail, caching, rate limiting, and resource isolation—to improve system reliability.

FailoverFault ToleranceOperations
0 likes · 14 min read
Understanding Faults and Fault Isolation Strategies in Distributed Systems
Java Backend Technology
Java Backend Technology
Aug 18, 2018 · Backend Development

Why Service Isolation Is Essential for Fault‑Tolerant Backend Systems

The article explains the concept of service isolation, its origins in shipbuilding, why it’s crucial for reducing fault impact in software systems, practical approaches such as functional and user‑based isolation, their trade‑offs, and key design principles to ensure reliable, maintainable back‑end architectures.

Backend ArchitectureFault ToleranceMicroservices
0 likes · 7 min read
Why Service Isolation Is Essential for Fault‑Tolerant Backend Systems
Meitu Technology
Meitu Technology
Aug 2, 2018 · Big Data

Spark Streaming vs Flink – Architecture, Scheduling & Fault Tolerance

This article compares Spark Streaming and Flink across runtime models, component roles, programming APIs, task scheduling, time semantics, dynamic Kafka partition detection, fault‑tolerance mechanisms, exactly‑once guarantees, and back‑pressure handling, providing code examples and practical insights for real‑time data processing.

Dynamic Partition DetectionFault ToleranceFlink
0 likes · 23 min read
Spark Streaming vs Flink – Architecture, Scheduling & Fault Tolerance
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Jul 20, 2018 · Backend Development

Mastering Service Discovery and Communication in Microservices

This article explains how microservices use service registries for discovery, registration, health checks, and deregistration, compares third‑party and self‑registration, explores server‑side and client‑side call mechanisms, discusses API gateways, synchronous vs asynchronous messaging, and outlines fault‑tolerance patterns such as timeouts, circuit breakers, and bulkheads.

API GatewayFault ToleranceMicroservices
0 likes · 21 min read
Mastering Service Discovery and Communication in Microservices
Architecture Digest
Architecture Digest
Jul 19, 2018 · Operations

How to Prevent System Failures: Suspect Third‑Party Services, Guard Consumers, and Strengthen Your Own Service

The article presents practical strategies for avoiding service failures by treating third‑party dependencies as unreliable, designing robust APIs for consumers, and applying solid engineering principles such as degradation plans, timeout settings, traffic control, and resource‑limiting techniques.

API DesignFault Tolerancereliability
0 likes · 16 min read
How to Prevent System Failures: Suspect Third‑Party Services, Guard Consumers, and Strengthen Your Own Service
ITPUB
ITPUB
Jun 6, 2018 · Cloud Native

How to Build a Cloud‑Native Microservices PaaS with Spring Cloud Netflix

This article explains how to construct a PaaS cloud platform using microservice architecture and Docker containers, detailing the roles of Spring Cloud Netflix components such as Zuul, Eureka, Hystrix, and Config Server, and covering gateway routing, service discovery, deployment, fault tolerance, and dynamic configuration.

Fault ToleranceMicroservicesSpring Cloud
0 likes · 13 min read
How to Build a Cloud‑Native Microservices PaaS with Spring Cloud Netflix
Meituan Technology Team
Meituan Technology Team
May 31, 2018 · Operations

High‑Availability Practices for Account Services at Meituan/Dianping

Meituan/Dianping ensures its critical account service stays online by combining real‑time business monitoring, circuit‑breaker‑driven graceful degradation, and active‑active cross‑region deployment with isolated dependencies, versioned data sync, and automated cache updates, dramatically extending MTBF while cutting MTTR and latency.

Data synchronizationFault ToleranceHigh Availability
0 likes · 13 min read
High‑Availability Practices for Account Services at Meituan/Dianping
Efficient Ops
Efficient Ops
May 27, 2018 · Operations

Mastering High Availability and High Concurrency: Principles and Practical Techniques

This article outlines guiding principles, high‑availability strategies, and high‑concurrency techniques—covering stateless design, resource isolation, quota management, monitoring, degradation, rollback, and scaling—to help engineers build resilient, scalable systems while balancing cost and performance.

Fault ToleranceOperationsSystem Design
0 likes · 21 min read
Mastering High Availability and High Concurrency: Principles and Practical Techniques
Efficient Ops
Efficient Ops
May 21, 2018 · Databases

Why Do Database Failures Happen and How to Prevent Them?

This article examines common hardware and network failures in data centers, analyzes real‑world outage cases, classifies fault domains, and presents comprehensive strategies for database fault handling—including logging, checkpointing, backup, replication, and high‑availability architectures—to improve reliability and reduce downtime.

BackupDatabaseFault Tolerance
0 likes · 22 min read
Why Do Database Failures Happen and How to Prevent Them?
ITFLY8 Architecture Home
ITFLY8 Architecture Home
May 19, 2018 · Backend Development

How to Structure Functional Teams and Service Patterns for Scalable Microservices

This article explains how Conway's law guides functional team division in microservice architectures, describes decentralized governance, outlines various interaction and composition patterns, discusses fault‑tolerance mechanisms such as isolation, circuit breaking, rate limiting, and provides guidance on choosing appropriate service granularity.

Fault ToleranceMicroservicesarchitecture
0 likes · 32 min read
How to Structure Functional Teams and Service Patterns for Scalable Microservices
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Mar 24, 2018 · Operations

How Service Degradation and Fault‑Tolerance Keep Large‑Scale Systems Resilient

This article explains how setting low timeouts for non‑core services, decoupling and physically isolating micro‑services, separating light and heavy workloads, and implementing automated configuration checks together enhance system reliability and reduce both technical and human errors in high‑traffic environments.

Fault Toleranceconfiguration managementsystem reliability
0 likes · 9 min read
How Service Degradation and Fault‑Tolerance Keep Large‑Scale Systems Resilient
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Mar 22, 2018 · Operations

How Simple Retry Can Crash Your System and Smarter Alternatives

This article examines the pitfalls of naive retry mechanisms, explores active‑standby service switching, dynamic removal of unhealthy nodes, proper timeout configuration, and anti‑reentrancy strategies to improve system reliability and prevent cascading failures in large‑scale backend operations.

Fault Toleranceretryservice reliability
0 likes · 14 min read
How Simple Retry Can Crash Your System and Smarter Alternatives
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Mar 4, 2018 · Operations

Mastering Service Fault Tolerance: Key Patterns for Resilient Microservices

Effective fault tolerance is crucial for microservice stability, and this article explores core design principles and classic patterns—such as timeout retries, rate limiting, bulkhead isolation, circuit breakers, and fallback strategies—guiding developers to choose and combine the right approaches for high‑availability systems.

BulkheadFault ToleranceMicroservices
0 likes · 8 min read
Mastering Service Fault Tolerance: Key Patterns for Resilient Microservices
Efficient Ops
Efficient Ops
Feb 23, 2018 · Operations

What a Decade of Ops Taught Me: Key Strategies for Scalable Infrastructure

This article reflects on ten years of Tencent's operations experience, sharing the author's career journey, the evolution of large‑scale service management, the design of the L5 fault‑tolerant system, unified frameworks, resource packaging, CMDB virtual mirrors, and automated deployment practices that together enable reliable, efficient, and scalable infrastructure.

AutomationCMDBFault Tolerance
0 likes · 11 min read
What a Decade of Ops Taught Me: Key Strategies for Scalable Infrastructure
Tencent TDS Service
Tencent TDS Service
Feb 1, 2018 · Backend Development

How a TV App’s Waterfall Layout Boosted User Engagement and Efficiency

This article details the redesign of a TV app from a horizontal layout to a waterfall flow, explaining the project timeline, advantages, new seven‑layer architecture, CMS‑driven configuration, compatibility handling, pagination strategies, caching, and fault‑tolerance measures that together improved user conversion and system robustness.

Backend ArchitectureCMSFault Tolerance
0 likes · 36 min read
How a TV App’s Waterfall Layout Boosted User Engagement and Efficiency
dbaplus Community
dbaplus Community
Jan 8, 2018 · Operations

From Firefighter to Automation: Tencent’s Ops Veteran Shares 10‑Year Infrastructure Secrets

Veteran Tencent operations leader Zhao Jianchun recounts a decade of managing 100,000 servers, detailing the L5 fault‑tolerant system, unified framework, resource packaging, CMDB virtual imaging, and an automated deployment platform that together cut daily incidents by up to 90% and boosted efficiency tenfold.

AutomationCMDBFault Tolerance
0 likes · 11 min read
From Firefighter to Automation: Tencent’s Ops Veteran Shares 10‑Year Infrastructure Secrets
21CTO
21CTO
Nov 20, 2017 · Operations

Mastering High Availability and Concurrency: Core Principles and Practical Techniques

This article distills essential guiding principles, high‑availability strategies, and high‑concurrency techniques for building resilient, scalable systems, covering stateless design, fault‑handling phases, replication, isolation, rate limiting, caching, async processing, multithreading, and scaling approaches.

Fault ToleranceHigh AvailabilitySystem Design
0 likes · 21 min read
Mastering High Availability and Concurrency: Core Principles and Practical Techniques
Efficient Ops
Efficient Ops
Nov 15, 2017 · Big Data

How Tencent Built a 10 TB‑Per‑Day Full‑Link Log Monitoring Platform

This article explains how Tencent's ZhiYun full‑link log monitoring platform handles massive daily logs, overcomes challenges of diverse log formats, high throughput, fault‑tolerant design, and provides scalable storage, query, and alerting capabilities for distributed micro‑service environments.

Big DataData PipelineFault Tolerance
0 likes · 10 min read
How Tencent Built a 10 TB‑Per‑Day Full‑Link Log Monitoring Platform
21CTO
21CTO
Oct 22, 2017 · Operations

How to Build Highly Available Systems: Fault Tolerance and Scalability Strategies

This article explains why high availability is critical for internet services, outlines key techniques such as stateless design, service discovery, heartbeat checks, idempotent operations, load balancing, throttling, caching, and micro‑service architecture, and discusses the operational challenges and monitoring tools needed to maintain resilient, scalable systems.

Fault ToleranceHigh AvailabilityIdempotency
0 likes · 8 min read
How to Build Highly Available Systems: Fault Tolerance and Scalability Strategies
Architecture Digest
Architecture Digest
Oct 15, 2017 · Operations

High Concurrency and High Availability Design Principles

This article outlines essential high‑concurrency and high‑availability principles—including stateless design, service decomposition, caching strategies, message queues, data heterogeneity, degradation, rate limiting, traffic switching, and rollback mechanisms—to help architects build scalable, reliable, and resilient systems.

Fault ToleranceHigh AvailabilitySystem Design
0 likes · 12 min read
High Concurrency and High Availability Design Principles
21CTO
21CTO
Sep 26, 2017 · Operations

Why You Should Never Trust Any Component in Your System—and How to Protect It

In programming and operations, every element—from services and dependencies to requests, machines, data centers, power, networks, and humans—can fail unexpectedly, so you must assume distrust and implement defensive measures such as monitoring, redundancy, rate limiting, fallback strategies, backups, and automated deployment.

Fault ToleranceMonitoringOperations
0 likes · 9 min read
Why You Should Never Trust Any Component in Your System—and How to Protect It
21CTO
21CTO
Aug 11, 2017 · Operations

Alibaba’s Double 11 Playbook: Scaling Architecture and Real‑Time Fault Tolerance

Alibaba’s eight‑year evolution of Double 11 showcases how limited cost can deliver maximal user experience and massive throughput by transitioning from a centralized 3.0 distributed architecture to multi‑active zones, employing capacity planning, full‑link stress testing, fine‑grained dependency governance, and dynamic traffic scheduling to ensure high availability.

Fault ToleranceLoad Balancingcapacity planning
0 likes · 12 min read
Alibaba’s Double 11 Playbook: Scaling Architecture and Real‑Time Fault Tolerance
Architecture Digest
Architecture Digest
Jul 16, 2017 · Operations

Fault Governance in Distributed Systems: Dependency Failures, Strong/Weak Dependency, and Fault‑Injection Practices

This article presents a comprehensive overview of fault governance in large‑scale distributed systems, covering classic dependency failures, the concept of strong and weak dependencies, experimental observations, the evolution of fault‑injection techniques, and best practices for building reliable fault‑drill platforms.

Chaos EngineeringDependency ManagementFault Tolerance
0 likes · 20 min read
Fault Governance in Distributed Systems: Dependency Failures, Strong/Weak Dependency, and Fault‑Injection Practices
Architecture Digest
Architecture Digest
Jul 6, 2017 · Fundamentals

PacificA: Microsoft’s General Replication Framework for Large‑Scale Distributed Storage Systems

PacificA is Microsoft’s generic replication framework for large‑scale distributed storage systems that provides strong consistency, separates configuration management from data replication, and uses a primary‑secondary model with lease‑based fault detection to ensure availability, correctness, and efficient operation across heterogeneous nodes.

Fault TolerancePacificAconsistency
0 likes · 14 min read
PacificA: Microsoft’s General Replication Framework for Large‑Scale Distributed Storage Systems
Suning Technology
Suning Technology
May 18, 2017 · Big Data

Why Apache Flink Beats Spark and Storm in Stream Processing

This article examines Apache Flink's stream‑processing architecture, compares its native streaming model, fault‑tolerance, performance and SQL capabilities with Spark and Storm, and concludes that Flink offers a more powerful and efficient solution despite some maturity gaps.

Apache FlinkFault ToleranceSpark
0 likes · 12 min read
Why Apache Flink Beats Spark and Storm in Stream Processing
Alibaba Cloud Developer
Alibaba Cloud Developer
May 12, 2017 · Operations

How Alibaba Engineers Fault Governance and Chaos Engineering for E‑commerce

This article recounts Alibaba's middleware team's QCon Beijing 2017 presentation on fault governance and fault‑drill practices, covering distributed‑system dependency failures, strong/weak dependency concepts, multi‑stage technical evolution, and the design of their chaos‑engineering platform for large‑scale e‑commerce.

AlibabaChaos EngineeringFault Tolerance
0 likes · 21 min read
How Alibaba Engineers Fault Governance and Chaos Engineering for E‑commerce
DevOps
DevOps
May 8, 2017 · Backend Development

Key Technical Concerns and Core Components of Microservices Architecture

Microservices architecture introduces technical concerns such as service registration, discovery, load balancing, health checks, front-end routing, fault tolerance, dynamic configuration, and framework selection, with common solutions ranging from centralized and in-process load balancers to Netflix and Spring Cloud components.

Fault ToleranceFrameworksLoad Balancing
0 likes · 16 min read
Key Technical Concerns and Core Components of Microservices Architecture
Alibaba Cloud Developer
Alibaba Cloud Developer
Apr 21, 2017 · Big Data

How Alibaba Tackles Real-Time Stream and Graph Computing at Scale

In his ASPLOS keynote, Alibaba’s Vice President Zhou Jingren detailed the company’s large‑scale stream and graph computing platforms, highlighting fault‑tolerance innovations, real‑time data challenges, and upcoming advances in graph analytics and massive machine‑learning workloads.

AIAlibabaBig Data
0 likes · 7 min read
How Alibaba Tackles Real-Time Stream and Graph Computing at Scale
Architecture Digest
Architecture Digest
Apr 16, 2017 · Operations

Common Load‑Balancing Strategies and Their Reliability Analysis in Distributed Systems

The article reviews hardware and software load‑balancing, explains classic strategies such as round‑robin, random, minimum‑response‑time, least‑connections and hash, and quantitatively evaluates their fault‑tolerance using probability formulas and example scenarios in distributed systems.

Fault ToleranceLeast ConnectionsLoad Balancing
0 likes · 10 min read
Common Load‑Balancing Strategies and Their Reliability Analysis in Distributed Systems
Qunar Tech Salon
Qunar Tech Salon
Feb 23, 2017 · Backend Development

Microservice Fault Tolerance: Timeout, Retry, Circuit Breaker, Rate Limiting, and Service Degradation

This article explains microservice fault‑tolerance techniques—including timeout settings, retry strategies, circuit‑breaker logic, current limiting, resource isolation, and service degradation—from both micro and macro perspectives, illustrating how to design resilient service chains and avoid cascading failures.

Fault Tolerancecircuit breakerrate limiting
0 likes · 11 min read
Microservice Fault Tolerance: Timeout, Retry, Circuit Breaker, Rate Limiting, and Service Degradation
Tencent Cloud Developer
Tencent Cloud Developer
Feb 14, 2017 · Databases

TDSQL Audit Capability: Architecture, Kafka Integration, and Consistency Hash Implementation

TDSQL’s cloud‑based audit solution combines a three‑proxy high‑availability layer, Kafka’s O(1) persistent messaging, and a distributed audit‑server that uses consistent hashing and multi‑coroutine processing to consume data within seconds, while fault‑tolerant offsets, majority acknowledgments, and Tencent Cloud MongoDB storage ensure secure, ordered, scalable, and highly reliable audit logging.

Fault ToleranceKafkaMongoDB
0 likes · 7 min read
TDSQL Audit Capability: Architecture, Kafka Integration, and Consistency Hash Implementation
Tencent Cloud Developer
Tencent Cloud Developer
Feb 9, 2017 · Backend Development

Backend Design and Implementation of QQ Game Spring Festival Red Packet System

The article details the QQ Game Spring Festival Red Packet backend, describing its multi‑phase architecture that handles 80 k RPS, uses CDN‑served static gift data, two‑level sorting, CMEM caching, RocketMQ buffering for throttled delivery, idempotent order tracking for fault tolerance, and unified real‑time monitoring.

Fault Toleranceasynchronous processingbackend
0 likes · 16 min read
Backend Design and Implementation of QQ Game Spring Festival Red Packet System
Efficient Ops
Efficient Ops
Feb 6, 2017 · Operations

Building Billion‑Scale Web Systems That Auto‑Extinguish Failures

The article shares Tencent’s practical fault‑tolerance journey for a billion‑scale activity platform, covering retry strategies, automatic removal of faulty nodes, timeout tuning, business‑level safeguards, service degradation, and decoupling techniques that together reduce manual firefighting and improve system resilience.

Fault ToleranceOperationslarge-scale systems
0 likes · 25 min read
Building Billion‑Scale Web Systems That Auto‑Extinguish Failures