Skip to content
Tagged articles

fault tolerance

355 articles · Page 3 of 4
21CTO
21CTO
Jul 16, 2021 · Operations

What Bilibili’s Outage Teaches About Achieving True High Availability

The article analyzes Bilibili’s recent service outage, explains why high availability matters, introduces key metrics like MTBF and MTTR, and outlines practical strategies such as redundancy, rate limiting, isolation, failover, timeout control, circuit breaking, degradation, and multi‑region deployment to build resilient systems.

Circuit BreakerHigh AvailabilityMTBF
0 likes · 18 min read
What Bilibili’s Outage Teaches About Achieving True High Availability
vivo Internet Technology
vivo Internet Technology
Jul 14, 2021 · Backend Development

Hystrix Source Code Analysis: Circuit Breaker, Isolation, and Fallback Mechanisms

Analyzing Hystrix’s source code reveals how its circuit‑breaker, bulkhead isolation (semaphore or thread‑pool), timeout detection, fallback logic, and sliding‑window health metrics work together to prevent cascading failures in distributed systems, as illustrated by an e‑commerce order service calling multiple downstream services.

Circuit BreakerHystrixNetflix OSS
0 likes · 20 min read
Hystrix Source Code Analysis: Circuit Breaker, Isolation, and Fallback Mechanisms
ITPUB
ITPUB
Jun 29, 2021 · Backend Development

Is Redis Distributed Lock Really Safe? A Deep Dive into Redlock, Pitfalls, and Alternatives

This article thoroughly examines the safety of Redis‑based distributed locks, explains basic SETNX locking, explores deadlock and lock‑release problems, presents robust solutions such as atomic SET with expiration, Lua scripts, and unique tokens, and critically compares Redlock with Zookeeper while summarizing expert debates and best‑practice recommendations.

LuaRedLockRedis
0 likes · 34 min read
Is Redis Distributed Lock Really Safe? A Deep Dive into Redlock, Pitfalls, and Alternatives
Architects Research Society
Architects Research Society
Jun 16, 2021 · Backend Development

Common Pitfalls in Microservice Integration and How to Mitigate Them

The article explains three frequent pitfalls when adopting microservices—complex remote communication, asynchronous processing challenges, and distributed transaction difficulties—and shows how fast‑fail, retries, timeouts, compensation, lightweight workflow engines, and idempotency can reduce complexity and improve resilience.

IdempotencyWorkflowdistributed systems
0 likes · 13 min read
Common Pitfalls in Microservice Integration and How to Mitigate Them
21CTO
21CTO
Jun 9, 2021 · Cloud Native

Baidu’s Low‑Intrusion, High‑Performance Service Mesh: Architecture & Lessons

This article details Baidu’s internal service‑mesh deployment, explaining why traditional RPC‑based governance fell short, how a sidecar‑based mesh decouples governance from frameworks, and the technical challenges and solutions for low‑intrusion, high‑performance, fault‑tolerant traffic management across tens of thousands of microservices.

Traffic Managementcloud-nativefault tolerance
0 likes · 18 min read
Baidu’s Low‑Intrusion, High‑Performance Service Mesh: Architecture & Lessons
Baidu Geek Talk
Baidu Geek Talk
Jun 9, 2021 · Cloud Native

Baidu's Internal Service Mesh Practice: Architecture, Challenges, and Performance Optimizations

Baidu created an internally‑built, Istio‑based service mesh that decouples governance from language‑specific RPCs, offering low‑intrusion integration, ultra‑low latency via a brpc coroutine data plane, advanced fault‑tolerance and fine‑grained traffic scheduling, and now powers over 80 % of its core microservices handling more than a trillion daily requests.

EnvoyIstioTraffic Management
0 likes · 17 min read
Baidu's Internal Service Mesh Practice: Architecture, Challenges, and Performance Optimizations
Programmer DD
Programmer DD
Jun 4, 2021 · Operations

Mastering Fault‑Tolerant Microservices: Patterns for Reliable Distributed Systems

This article explores essential patterns and techniques—such as graceful degradation, change management, health checks, failover caching, retry logic, rate limiting, circuit breakers, and chaos testing—to build highly available microservice architectures that can withstand network, hardware, and application failures.

Circuit Breakercloud-nativefault tolerance
0 likes · 15 min read
Mastering Fault‑Tolerant Microservices: Patterns for Reliable Distributed Systems
Amap Tech
Amap Tech
May 28, 2021 · Operations

System Observability Practices in Gaode Ride-Hailing: From Unified Logging to Fault Defense

Gaode Ride‑Hailing created a comprehensive 360° observability platform—standardized logging, distributed tracing, multi‑domain metrics, visual dashboards, and an incident workflow—that transforms raw data into actionable insights, accelerates root‑cause analysis, and enables automated fault defense for its large‑scale cloud‑native microservice system.

LoggingTracingdistributed systems
0 likes · 22 min read
System Observability Practices in Gaode Ride-Hailing: From Unified Logging to Fault Defense
Architects' Tech Alliance
Architects' Tech Alliance
May 19, 2021 · Operations

Designing Microservices Architecture for Failure: Patterns and Practices

Microservice architectures must handle inevitable network, hardware, and application errors by employing fault‑tolerant patterns such as graceful degradation, change management, health checks, fail‑over caches, retry logic, rate limiting, circuit breakers, and testing strategies to maintain service reliability and user experience.

Operationsfault tolerancemicroservices
0 likes · 15 min read
Designing Microservices Architecture for Failure: Patterns and Practices
JD Tech Talk
JD Tech Talk
May 17, 2021 · Databases

Design and Optimization of Multi‑Data‑Center Redis Synchronization

This article describes the challenges of native Redis in multi‑data‑center deployments and presents the design, implementation, and performance evaluation of a custom Redis extension that adds bidirectional synchronization, rlog logging, protocol enhancements, and conflict‑resolution mechanisms to achieve reliable cross‑region active‑active operation.

RLogdata synchronizationfault tolerance
0 likes · 16 min read
Design and Optimization of Multi‑Data‑Center Redis Synchronization
Top Architect
Top Architect
Apr 21, 2021 · Backend Development

Microservice Architecture Evolution: From Monolith to Service Mesh

This article chronicles the evolution of an online supermarket from a simple monolithic website to a fully split microservice architecture, discussing the motivations, challenges, design patterns, monitoring, fault tolerance, testing, service discovery, and the eventual adoption of service mesh.

Backendarchitecturefault tolerance
0 likes · 23 min read
Microservice Architecture Evolution: From Monolith to Service Mesh
vivo Internet Technology
vivo Internet Technology
Apr 14, 2021 · Fundamentals

An Overview of the Raft Distributed Consensus Algorithm

Raft is a fault‑tolerant distributed consensus protocol that simplifies Paxos by electing a single leader each term to coordinate client requests, replicate logs to a majority of servers, ensure safety through up‑to‑date voting, handle failures with randomized timeouts, resolve log conflicts, and compress logs via snapshots.

Distributed ConsensusLeader ElectionLog Replication
0 likes · 19 min read
An Overview of the Raft Distributed Consensus Algorithm
Kuaishou Tech
Kuaishou Tech
Apr 9, 2021 · Backend Development

Design and Implementation of Red Packet Calculation and Distribution System for Spring Festival Activity

This article details the design of a red packet calculation and distribution system for a Spring Festival promotion, addressing mixed-type packet computation, seamless transition to awarding, distributed task processing, stability guarantees, and successful implementation results.

Spring Festivalbatch processingdistributed computing
0 likes · 12 min read
Design and Implementation of Red Packet Calculation and Distribution System for Spring Festival Activity
IT Architects Alliance
IT Architects Alliance
Apr 5, 2021 · Operations

Design and Implementation of a Handcrafted Distributed Cluster (MyCluster)

This article describes how to design and build a native distributed cluster called MyCluster without using any existing frameworks, covering master‑slave architecture, leader election, split‑brain handling, centralized configuration management, custom communication protocols, state transitions, and client interfaces.

Cluster ArchitectureConfiguration ManagementLeader Election
0 likes · 13 min read
Design and Implementation of a Handcrafted Distributed Cluster (MyCluster)
DataFunTalk
DataFunTalk
Mar 28, 2021 · Big Data

Flink Stream‑Batch Integration: Layered Architecture, Unified SDK, DAG Scheduler, Shuffle, and Fault‑Tolerance

This article explains how Apache Flink has evolved into a unified stream‑batch engine by introducing a three‑layer architecture, a unified DataStream SDK, a pipeline‑region‑based DAG scheduler, a common shuffle framework, and enhanced fault‑tolerance mechanisms to address efficiency, consistency, and resource‑utilisation challenges in real‑time big‑data processing.

Apache FlinkDAG schedulerShuffle Architecture
0 likes · 25 min read
Flink Stream‑Batch Integration: Layered Architecture, Unified SDK, DAG Scheduler, Shuffle, and Fault‑Tolerance
Baidu Geek Talk
Baidu Geek Talk
Mar 22, 2021 · Operations

How Baidu Achieved 99.999% Uptime for Its Massive Feed Recommendation System

This article details Baidu's Feed recommendation system architecture, explaining how a combination of dynamic retry scheduling, real‑time stop‑loss mechanisms, multi‑recall frameworks, ranking layer fallbacks, and IDC‑level multi‑master designs collectively ensure five‑nine availability across billions of daily requests.

High AvailabilityOperationsRecommendation System
0 likes · 18 min read
How Baidu Achieved 99.999% Uptime for Its Massive Feed Recommendation System
IT Architects Alliance
IT Architects Alliance
Mar 15, 2021 · Backend Development

Evolution of Meituan Instant Logistics Distributed System Architecture and Practices

The article details Meituan's five‑year journey in instant logistics, describing how distributed, high‑concurrency backend architectures were progressively upgraded to microservices, how AI is integrated for pricing, ETA and dispatch, and the operational techniques used to ensure scalability, fault tolerance, and high availability.

AIMeituandistributed systems
0 likes · 8 min read
Evolution of Meituan Instant Logistics Distributed System Architecture and Practices
Xianyu Technology
Xianyu Technology
Feb 8, 2021 · Backend Development

Design and Implementation of a Cluster‑Aware Guava Cache Component for High Reliability

The paper presents a cluster‑aware Guava cache component for Alibaba’s Xianyu platform that mitigates downstream service failures by adding asynchronous reload, cluster‑wide key invalidation, and size reporting, enabling automatic fallback to refreshed local data and improving latency, with future plans for a management console, tiered storage, and disk‑backed caching.

CachingGuavafault tolerance
0 likes · 8 min read
Design and Implementation of a Cluster‑Aware Guava Cache Component for High Reliability
Sohu Tech Products
Sohu Tech Products
Jan 20, 2021 · Backend Development

Understanding Dubbo’s Core Architecture: Service Registration, Invocation, Routing, and Thread Dispatch Mechanisms

This article explains Dubbo’s internal architecture, covering service registration and discovery with Zookeeper, RPC invocation details including load balancing, routing, and fault‑tolerance strategies, as well as its network protocol and thread‑dispatch mechanisms, providing practical insights for backend developers.

DubboRoutingService Discovery
0 likes · 13 min read
Understanding Dubbo’s Core Architecture: Service Registration, Invocation, Routing, and Thread Dispatch Mechanisms
Top Architect
Top Architect
Jan 6, 2021 · Cloud Native

Implementing a Microservice Architecture with Spring Cloud, Docker, and PaaS

This article explains how to build a microservice‑based PaaS platform using Spring Cloud Netflix components, Docker containers, Eureka for service registration, Zuul as a gateway, Hystrix for fault tolerance, and a dynamic configuration center to achieve agile development and continuous integration.

DockerDynamic ConfigurationGateway
0 likes · 13 min read
Implementing a Microservice Architecture with Spring Cloud, Docker, and PaaS
Efficient Ops
Efficient Ops
Jan 5, 2021 · Operations

How to Prevent ZooKeeper Split‑Brain: Best Practices and Fault‑Tolerance Strategies

This article explains why ZooKeeper clusters should use an odd number of nodes, how the majority quorum mechanism avoids split‑brain scenarios, and outlines practical solutions such as quorums, redundant communication, fencing, arbitration, and disk‑lock techniques to ensure reliable distributed coordination.

Split-BrainZooKeeperdistributed systems
0 likes · 14 min read
How to Prevent ZooKeeper Split‑Brain: Best Practices and Fault‑Tolerance Strategies
Architects Research Society
Architects Research Society
Dec 30, 2020 · Fundamentals

Key Challenges in Designing Distributed Systems

Designing a distributed system involves overcoming major challenges such as heterogeneity, transparency, openness, concurrency, security, scalability, and fault tolerance, each requiring careful consideration of hardware, software, network, and management aspects to build robust, scalable, and secure architectures.

Securitydistributed systemsfault tolerance
0 likes · 9 min read
Key Challenges in Designing Distributed Systems
FunTester
FunTester
Dec 12, 2020 · Operations

Why Redundancy Is the Key to Effective Disaster Recovery in IT Systems

The article explains that disaster recovery for information systems relies on redundancy across hardware, energy, and data, classifies natural, human, and technical disasters, defines critical metrics such as RTO and RPO, and outlines the technologies, architectures, and maturity levels needed to ensure business continuity.

RPORTObusiness continuity
0 likes · 29 min read
Why Redundancy Is the Key to Effective Disaster Recovery in IT Systems
Architecture Digest
Architecture Digest
Dec 9, 2020 · Backend Development

Implementing Distributed Locks with Redis: Concepts, Algorithms, and Code Examples

This article explains how to implement distributed locks using Redis, covering the essential requirements of mutual exclusion, deadlock avoidance, and fault tolerance, detailing single‑instance and multi‑instance algorithms, code examples with SETNX and Lua scripts, and discussing challenges such as latency, crashes, and persistence.

Concurrencybackend developmentfault tolerance
0 likes · 10 min read
Implementing Distributed Locks with Redis: Concepts, Algorithms, and Code Examples
Manbang Technology Team
Manbang Technology Team
Nov 23, 2020 · Operations

Designing a Comprehensive Stability Assurance System for Large‑Scale Internet Services at Manbang

This article explains how Manbang built a rigorous stability‑assurance framework—including strict fault grading, a "watch‑and‑protect" system, blue‑green deployments, online pressure testing, fault‑drill platforms, and runtime metadata—to ensure rapid iteration while maintaining high availability for millions of logistics users.

fault tolerance
0 likes · 12 min read
Designing a Comprehensive Stability Assurance System for Large‑Scale Internet Services at Manbang
Tencent Cloud Developer
Tencent Cloud Developer
Nov 19, 2020 · Backend Development

Kafka Message Queue Reliability Design and Implementation

The article thoroughly explains Kafka’s message‑queue reliability design and implementation, covering use‑case scenarios, core concepts, storage format, producer acknowledgment settings, broker replication mechanisms (ISR, HW, LEO), consumer delivery semantics, the epoch solution for synchronization, and practical configuration guidelines for various consistency and availability requirements.

ConsistencyConsumerKafka
0 likes · 15 min read
Kafka Message Queue Reliability Design and Implementation
JavaEdge
JavaEdge
Oct 24, 2020 · Databases

Mastering Redis Cluster: Scaling, Routing, and Fault Tolerance Explained

This article explains why Redis clusters are needed, how CLUSTER MEET builds the network, slot assignment, scaling procedures, client redirection mechanisms, batch operations, fault detection, recovery processes, and common operational pitfalls, providing practical guidance for building and maintaining a robust Redis Cluster deployment.

ClusterRedisfault tolerance
0 likes · 22 min read
Mastering Redis Cluster: Scaling, Routing, and Fault Tolerance Explained
DevOps
DevOps
Oct 20, 2020 · Cloud Computing

Chaos Monkey and the Simian Army: Building Resilient Cloud Systems

The article explains how Netflix uses Chaos Monkey and a suite of related tools, collectively called the Simian Army, to deliberately inject failures into their cloud infrastructure, continuously test fault‑tolerance, and ensure high availability and reliability for their streaming service.

NetflixOperationsSimian Army
0 likes · 7 min read
Chaos Monkey and the Simian Army: Building Resilient Cloud Systems
IT Architects Alliance
IT Architects Alliance
Oct 13, 2020 · Cloud Native

Designing Fault‑Tolerant Microservices Architecture

Microservice architectures increase system complexity and failure rates, so this article explains key reliability patterns—such as graceful degradation, change management, health checks, self‑healing, fallback caches, retry logic, rate limiting, circuit breakers, and testing—to help engineers design resilient, high‑availability services.

Circuit BreakerOperationscloud-native
0 likes · 23 min read
Designing Fault‑Tolerant Microservices Architecture
Architects' Tech Alliance
Architects' Tech Alliance
Oct 12, 2020 · Operations

Designing Resilient Microservices: Patterns for Fault Tolerance and Failure Management

This article examines the inherent risks of microservice architectures and presents practical patterns—such as graceful degradation, change management, health checks, self‑healing, fallback caching, retries, rate limiting, bulkheads, and circuit breakers—to build highly available, fault‑tolerant services.

Circuit BreakerResiliencebulkhead
0 likes · 15 min read
Designing Resilient Microservices: Patterns for Fault Tolerance and Failure Management
Top Architect
Top Architect
Oct 11, 2020 · Cloud Native

Using Hystrix for Fault Tolerance in Spring Cloud Microservices

This article explains how to integrate Netflix Hystrix into Spring Cloud applications to provide request timeout, circuit‑breaker, fallback, monitoring and resource isolation for microservice calls, including Maven setup, annotation usage, Feign client fallback configuration and disabling options.

Circuit BreakerFeignHystrix
0 likes · 9 min read
Using Hystrix for Fault Tolerance in Spring Cloud Microservices
Architect
Architect
Oct 6, 2020 · Backend Development

Implementing Hystrix for Fault Tolerance in Spring Cloud Microservices

This article explains why microservice calls need fault‑tolerance mechanisms, introduces Hystrix’s core features such as timeouts, circuit‑breaker, fallback, monitoring and resource isolation, and provides step‑by‑step code examples for integrating Hystrix and Feign in a Spring Cloud project.

FeignHystrixJava
0 likes · 8 min read
Implementing Hystrix for Fault Tolerance in Spring Cloud Microservices
DataFunTalk
DataFunTalk
Oct 2, 2020 · Big Data

Single-Task Recovery in Flink: Design and Implementation for Real‑Time Stream Processing

This article describes ByteDance's single‑task recovery solution for Flink's real‑time computation, detailing the problem of global job restarts, the proposed network‑layer enhancements, upstream and downstream optimizations, JobManager restart strategy, implementation challenges, and the measurable latency and availability benefits achieved in production.

FlinkSingle-Task Recoveryfault tolerance
0 likes · 11 min read
Single-Task Recovery in Flink: Design and Implementation for Real‑Time Stream Processing
Xianyu Technology
Xianyu Technology
Sep 27, 2020 · Backend Development

Design of an Asynchronous Component with Monitoring, Fault Tolerance, and Zero‑Cost Integration

The article presents a design for an asynchronous component that is monitorable, fault‑tolerant, and integrates with zero overhead, compares Akka, RxJava, and a custom JUC‑based implementation, and selects the latter—using extended Callables and a CountDownLatch—to track business units, handle timeouts, and provide fallback behavior.

ConcurrencyJUCJava
0 likes · 8 min read
Design of an Asynchronous Component with Monitoring, Fault Tolerance, and Zero‑Cost Integration
Architect's Tech Stack
Architect's Tech Stack
Sep 21, 2020 · Backend Development

Overview of Tars: A High‑Performance RPC Framework and Service Governance Platform

The article introduces Tars, an open‑source high‑performance RPC framework and integrated service governance platform derived from Tencent's internal microservice architecture, detailing its design philosophy, layered architecture, core features such as the Tars protocol, load balancing, fault and overload protection, and centralized configuration management.

RPCService GovernanceTars
0 likes · 11 min read
Overview of Tars: A High‑Performance RPC Framework and Service Governance Platform
21CTO
21CTO
Sep 12, 2020 · Fundamentals

Why Distributed Systems Mirror Single‑Node Concurrency and How to Avoid Common Pitfalls

This article explains how concurrency issues that appear in single‑threaded programs become amplified in distributed systems, covering consistency models, network reliability, clock synchronization, fault detection, backpressure, and cascading failures, and offers practical design and testing strategies to build resilient architectures.

ConcurrencyConsistencyfault tolerance
0 likes · 19 min read
Why Distributed Systems Mirror Single‑Node Concurrency and How to Avoid Common Pitfalls
Sohu Tech Products
Sohu Tech Products
Sep 2, 2020 · Backend Development

Implementing Distributed Locks with Redis and Redisson: Abstraction, Auto-Release, and Fault Tolerance

This article explains how to use Redis and Redisson for distributed locking, introduces an abstract DistributedLock interface for flexible implementations, demonstrates automatic lock release with functional callbacks, and discusses fallback strategies and monitoring to ensure reliability in backend systems.

Distributed LockJavaRedis
0 likes · 5 min read
Implementing Distributed Locks with Redis and Redisson: Abstraction, Auto-Release, and Fault Tolerance
Programmer DD
Programmer DD
Jul 31, 2020 · Fundamentals

What Is Distributed Architecture and Why It Powers Modern Systems

This article explains the concept of distributed architecture, its evolution from monolithic systems, core design principles, common challenges such as network latency and data consistency, and how major tech companies adopt it to achieve high availability, scalability, and fault tolerance.

fault tolerancesystem design
0 likes · 11 min read
What Is Distributed Architecture and Why It Powers Modern Systems
Programmer DD
Programmer DD
Jul 28, 2020 · Fundamentals

What Makes Distributed Architecture Essential for Modern Systems?

Distributed architecture, built on distributed computing technologies like J2EE, transforms monolithic systems into multi‑layer, fault‑tolerant platforms by decoupling services, ensuring high availability, scalability, and resilience, while addressing challenges such as network latency, data consistency, and system complexity, as illustrated by real‑world case studies.

case studyfault tolerancesystem design
0 likes · 11 min read
What Makes Distributed Architecture Essential for Modern Systems?
Qunar Tech Salon
Qunar Tech Salon
Jun 16, 2020 · Operations

Qunar's Multi-IDC Deployment and Fault Self‑Healing Architecture

This article describes how Qunar scaled its IDC infrastructure, introduced multi‑IDC deployment, automated DNS‑based load balancing, open‑source DNSDB, and an IDC proxy built on Squid to achieve rapid fault self‑healing and transparent traffic switching for both user and third‑party access.

DNSOperationsProxy
0 likes · 8 min read
Qunar's Multi-IDC Deployment and Fault Self‑Healing Architecture
Top Architect
Top Architect
Jun 10, 2020 · Fundamentals

A Comprehensive Guide to Learning Distributed Systems

This article provides a thorough overview of distributed systems, explaining their definition, core challenges, key characteristics, essential components, common protocols, and practical implementations to help readers build a solid, structured learning path for mastering distributed architectures.

distributed systemsfault tolerancesystem design
0 likes · 16 min read
A Comprehensive Guide to Learning Distributed Systems
Xiaokun's Architecture Exploration Notes
Xiaokun's Architecture Exploration Notes
May 29, 2020 · Fundamentals

How the Byzantine Generals Problem Shapes Modern Distributed Consensus

This article explains the Byzantine Generals Problem, maps its concepts to distributed consensus, distinguishes consensus from consistency, outlines oral‑message and signed‑message solutions, and analyzes their applicability and limitations in fault‑tolerant distributed systems.

Byzantine GeneralsDistributed Consensusconsensus algorithms
0 likes · 21 min read
How the Byzantine Generals Problem Shapes Modern Distributed Consensus
58 Tech
58 Tech
May 22, 2020 · Backend Development

Design and Implementation of a Distributed Retry System Based on Distributed Scheduling

This article presents a comprehensive distributed retry system that leverages a distributed scheduling mechanism to ensure eventual consistency, reduce manual recovery costs, and provide flexible retry strategies, automatic recovery detection, visual management, rate limiting, and intelligent retry for backend services.

backend developmentfault toleranceintelligent retry
0 likes · 13 min read
Design and Implementation of a Distributed Retry System Based on Distributed Scheduling
Big Data Technology & Architecture
Big Data Technology & Architecture
Apr 13, 2020 · Fundamentals

Understanding Replication, Consistency, Fault Tolerance, and the CAP Theorem in Distributed Systems

This article explains the core concepts of replication, consistency, and fault tolerance in distributed systems, discusses strong and asynchronous replication methods, and details the CAP theorem with its consistency, availability, and partition tolerance trade‑offs, illustrating AP and CP scenarios such as Eureka and Zookeeper clusters.

CAP theoremConsistencyfault tolerance
0 likes · 7 min read
Understanding Replication, Consistency, Fault Tolerance, and the CAP Theorem in Distributed Systems
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Mar 23, 2020 · Fundamentals

15 Timeless Architecture Principles Every Engineer Should Follow

This article outlines how to create solid software architectures by presenting a process for forming design principles, detailing fifteen universal architecture guidelines, and explaining service‑splitting and key design rules that together help build scalable, maintainable, and resilient systems.

Design Principlesfault tolerancemicroservices
0 likes · 16 min read
15 Timeless Architecture Principles Every Engineer Should Follow
360 Tech Engineering
360 Tech Engineering
Mar 10, 2020 · Fundamentals

Introduction to Raft: A Comprehensive Overview of the Distributed Consensus Algorithm

This article provides a thorough introduction to the Raft consensus algorithm, explaining its purpose, core components such as state machine replication, log and consensus module, leader‑follower model, client interaction, fault‑tolerance considerations, the CAP trade‑off, and why Go is a suitable implementation language.

Distributed ConsensusGoRaft
0 likes · 11 min read
Introduction to Raft: A Comprehensive Overview of the Distributed Consensus Algorithm
Youzan Coder
Youzan Coder
Feb 28, 2020 · Big Data

Flink Checkpoint Principle Analysis and Failure Cause Investigation

The article thoroughly explains Apache Flink’s checkpoint mechanism—including state types, coordinator workflow, exactly‑once versus at‑least‑once semantics, common failure sources such as code exceptions, storage or network issues, and practical configuration tips like interval settings, local recovery and externalized checkpoints.

Apache FlinkReal-time ComputingState Management
0 likes · 15 min read
Flink Checkpoint Principle Analysis and Failure Cause Investigation
Architects' Tech Alliance
Architects' Tech Alliance
Feb 10, 2020 · Fundamentals

Mastering Distributed System Fundamentals: Models, Replication, Consistency, and Protocols

This article provides a comprehensive overview of distributed system fundamentals, covering node modeling, replica concepts, consistency levels, data distribution strategies, centralized and decentralized replica protocols, lease mechanisms, quorum, two‑phase commit, MVCC, Paxos, and the CAP theorem, while analyzing their trade‑offs in availability, consistency, and partition tolerance.

Consensus ProtocolsConsistencyData Distribution
0 likes · 55 min read
Mastering Distributed System Fundamentals: Models, Replication, Consistency, and Protocols
Architects' Tech Alliance
Architects' Tech Alliance
Feb 4, 2020 · Backend Development

Microservice Architecture Evolution: From Monolith to Service Mesh

This article walks through the transformation of an online supermarket from a simple monolithic website to a fully fledged microservice architecture, highlighting the motivations, design decisions, common pitfalls, and essential components such as monitoring, tracing, logging, gateways, service discovery, circuit breaking, testing strategies, and service mesh adoption.

architecturedeploymentfault tolerance
0 likes · 22 min read
Microservice Architecture Evolution: From Monolith to Service Mesh
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Jan 7, 2020 · Backend Development

Spring Cloud vs Dubbo: Protocol Handling, Performance, Load Balancing, Fault Tolerance, and Routing in Microservice Architecture

This article compares Spring Cloud and Dubbo across protocol handling, performance tuning, load‑balancing strategies, fault‑tolerance mechanisms, and routing/traffic‑shaping features, highlighting their flexibility, configuration complexity, and suitability for different microservice scenarios.

DubboSpring Cloudfault tolerance
0 likes · 6 min read
Spring Cloud vs Dubbo: Protocol Handling, Performance, Load Balancing, Fault Tolerance, and Routing in Microservice Architecture
Efficient Ops
Efficient Ops
Dec 17, 2019 · Operations

How Alibaba Scales Flink: Lessons in Big Data Operations

This article details Alibaba's massive Flink deployment, covering its historical background, the operational challenges of managing tens of thousands of nodes, the design of a comprehensive Flink management platform, and the automated solutions for fault handling, resource allocation, and performance testing in a large‑scale big‑data environment.

Flinkautomationbig data operations
0 likes · 20 min read
How Alibaba Scales Flink: Lessons in Big Data Operations
21CTO
21CTO
Nov 15, 2019 · Operations

How SRE Designs Highly Available Software Systems at Scale

This article presents Google SRE expert Ramón Medrano Llamas’s comprehensive guide on designing, operating, and maintaining large‑scale, highly available software systems, covering SRE fundamentals, daily workflows, scalability strategies, fault‑tolerant architecture, monitoring, and operational best practices.

SREfault tolerancemonitoring
0 likes · 13 min read
How SRE Designs Highly Available Software Systems at Scale
Java High-Performance Architecture
Java High-Performance Architecture
Nov 12, 2019 · Backend Development

How Kafka Consumer Groups Boost Performance and Fault Tolerance

Kafka consumer groups enable multiple consumers to share partition workloads, ensuring exclusive consumption within a group, flexible consumption patterns like broadcast and unicast, and automatic fault‑tolerance through rebalancing, ultimately improving throughput, scalability, and resilience of streaming applications.

Kafkabackend developmentconsumer groups
0 likes · 4 min read
How Kafka Consumer Groups Boost Performance and Fault Tolerance
Architecture Digest
Architecture Digest
Nov 9, 2019 · Backend Development

Design and Implementation of eBay's Next‑Generation Million‑TPS Core Accounting System

The article details eBay's 2018‑2020 design, performance testing, and fault‑tolerance architecture of a next‑generation core accounting system capable of handling millions of transactions per second, covering system goals, multi‑region deployment, event‑sourcing, Raft consensus, scalability optimizations, and the planned open‑source release.

High TPSRaft Consensusdistributed architecture
0 likes · 24 min read
Design and Implementation of eBay's Next‑Generation Million‑TPS Core Accounting System
Architects' Tech Alliance
Architects' Tech Alliance
Oct 2, 2019 · Operations

Understanding Disaster Tolerance, Fault Tolerance, and Disaster Recovery: Concepts, Differences, and Implementation Strategies

This article explains the definitions of disaster tolerance, fault tolerance, and disaster recovery, compares their purposes, discusses backup versus disaster‑tolerance solutions, outlines key metrics such as RTO and RPO, and presents common architectural and investment considerations for building resilient enterprise systems.

IT OperationsRPORTO
0 likes · 8 min read
Understanding Disaster Tolerance, Fault Tolerance, and Disaster Recovery: Concepts, Differences, and Implementation Strategies
Alibaba Cloud Native
Alibaba Cloud Native
Sep 25, 2019 · Cloud Native

Mastering Distributed System Design: Patterns, Performance, and Fault Tolerance

This article provides a comprehensive overview of distributed system architecture, covering essential design patterns such as gateways, sidecars, and service meshes, performance techniques like caching and async communication, fault‑tolerance mechanisms including rate limiting and circuit breakers, and practical DevOps practices for deployment and monitoring.

Architecture DesignCachingGateway
0 likes · 13 min read
Mastering Distributed System Design: Patterns, Performance, and Fault Tolerance
Architecture Digest
Architecture Digest
Sep 23, 2019 · Operations

Improving Application Availability: Practices, Monitoring, and Fault‑Tolerance in a Large‑Scale Payment System

The article describes how a high‑traffic payment platform achieves 99.999% availability by avoiding single points of failure, applying fail‑fast principles, implementing resource limits, building real‑time monitoring and alerting, and automating fault detection, routing, and recovery to ensure continuous 7×24 operation.

High Availabilitybackend operationsfault tolerance
0 likes · 23 min read
Improving Application Availability: Practices, Monitoring, and Fault‑Tolerance in a Large‑Scale Payment System
Java Captain
Java Captain
Sep 19, 2019 · Backend Development

A Comprehensive Overview of Microservice Architecture and Its Evolution

This article presents a detailed, step‑by‑step illustration of microservice architecture, covering its motivations, component breakdown, migration from monoliths, common pitfalls, monitoring, tracing, logging, gateway, service discovery, resilience patterns, testing strategies, frameworks, and the emerging service‑mesh approach.

Tracingfault tolerancemonitoring
0 likes · 23 min read
A Comprehensive Overview of Microservice Architecture and Its Evolution
AntTech
AntTech
Sep 11, 2019 · Artificial Intelligence

ElasticDL: An Open‑Source Elastic Deep Learning System Built on TensorFlow 2.0 and Kubernetes

ElasticDL, the first industry‑level open‑source system for elastic deep learning on TensorFlow, leverages Kubernetes‑native scheduling, fault‑tolerance, and TensorFlow 2.0 Eager Execution to dramatically improve cluster utilization, simplify distributed training, and integrate seamlessly with tools like Kubeflow and SQLFlow.

Distributed Deep LearningElasticDLKubernetes
0 likes · 13 min read
ElasticDL: An Open‑Source Elastic Deep Learning System Built on TensorFlow 2.0 and Kubernetes
dbaplus Community
dbaplus Community
Sep 10, 2019 · Big Data

Why Exactly‑Once Processing Is So Hard in Distributed Systems (And How to Tackle It)

This article explores the two toughest problems in distributed stream processing—exactly‑once message handling and ordering—by dissecting the underlying impossibility of perfect failure detectors, the liveness‑vs‑safety trade‑off, zombie processes, and the practical solutions employed by systems such as Flink, Kafka Streams, MillWheel, and Spark.

Consensuscheckpointingdistributed systems
0 likes · 81 min read
Why Exactly‑Once Processing Is So Hard in Distributed Systems (And How to Tackle It)
Big Data Technology & Architecture
Big Data Technology & Architecture
Jun 19, 2019 · Big Data

Understanding Spark Structured Streaming StateStore: Architecture, Operations, and Fault Recovery

This article explains the design and implementation of Spark Structured Streaming's StateStore module, covering its distributed architecture, state sharding, versioning, batch read/write, migration, update/query APIs, maintenance compaction, and fault‑tolerance mechanisms that enable incremental continuous queries with exactly‑once guarantees.

SparkStateStoreStructured Streaming
0 likes · 8 min read
Understanding Spark Structured Streaming StateStore: Architecture, Operations, and Fault Recovery
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Jun 5, 2019 · Fundamentals

Understanding Paxos: A Beginner’s 30‑Minute Guide with Real‑World Analogy

This article explains the Paxos consensus algorithm in plain terms, using a relatable travel‑planning analogy to illustrate how proposers, acceptors, and majority voting achieve fault‑tolerant agreement in distributed systems, and connects the concept to real‑world implementations like Google’s Chubby and ZooKeeper.

AlgorithmDistributed ConsensusPaxos
0 likes · 13 min read
Understanding Paxos: A Beginner’s 30‑Minute Guide with Real‑World Analogy
dbaplus Community
dbaplus Community
May 25, 2019 · Backend Development

Mastering Thread‑Pool Isolation: Prevent Cascading Failures in Java Services

This article explains the concept of fault tolerance in software architecture, illustrates why thread‑pool isolation is essential for preventing cascading failures, and provides concrete Java implementations—including code examples, pros and cons, and practical guidance for applying the technique in real‑world backend systems.

BackendJavafault tolerance
0 likes · 10 min read
Mastering Thread‑Pool Isolation: Prevent Cascading Failures in Java Services
21CTO
21CTO
Apr 29, 2019 · Big Data

How EasyScheduler Powers Scalable Big Data Workflow Management

EasyScheduler is an open‑source big‑data workflow scheduler that uses a decentralized architecture with Master and Worker nodes coordinated via ZooKeeper, supporting DAG‑based task definitions, various task types, fault tolerance, priority handling, distributed locks, and remote logging, all illustrated with detailed component diagrams.

DAGWorkflowbig data
0 likes · 17 min read
How EasyScheduler Powers Scalable Big Data Workflow Management
Architecture Digest
Architecture Digest
Apr 29, 2019 · Big Data

EasyScheduler: An Open‑Source Big Data Workflow Scheduling System – Architecture and Design Overview

This article introduces EasyScheduler, an open‑source big data workflow scheduling system, explaining its core terminology, decentralized architecture, distributed lock implementation, thread‑shortage handling, fault‑tolerance mechanisms, task‑retry and priority designs, as well as its logging solution using Logback and gRPC.

DAGLoggingWorkflow
0 likes · 14 min read
EasyScheduler: An Open‑Source Big Data Workflow Scheduling System – Architecture and Design Overview
ITPUB
ITPUB
Mar 26, 2019 · Operations

How to Build a 99.99% High‑Availability Service: Practices and Architecture Evolution

This article explains the essential requirements for achieving 99.99% service availability—consistency, eliminating single points, placement groups, traffic isolation, same‑city active‑active, N+1 redundancy, and multi‑region active‑active—illustrated with a step‑by‑step Yum repository service case study and evolving architecture diagrams.

Cloud Operationsarchitecturedeployment
0 likes · 9 min read
How to Build a 99.99% High‑Availability Service: Practices and Architecture Evolution
iQIYI Technical Product Team
iQIYI Technical Product Team
Mar 15, 2019 · Cloud Computing

Design and Architecture of QLive Large‑Scale Live Streaming Service

The QLive service powers iQIYI’s massive live‑streaming events—such as the Spring Festival Gala—by combining vertical and horizontal scaling, a three‑layer architecture with dual data‑center isolation, multi‑level caching, circuit‑breaker/degradation controls, and a Flume‑Kafka‑Hive monitoring pipeline to sustain over 400 k QPS and 99.9999 % availability.

CachingHorizontal Scalingfault tolerance
0 likes · 9 min read
Design and Architecture of QLive Large‑Scale Live Streaming Service
Big Data Technology & Architecture
Big Data Technology & Architecture
Mar 13, 2019 · Big Data

Understanding Fault Tolerance and Exactly-Once Semantics in Apache Flink

This article explains Apache Flink's fault‑tolerance mechanisms, including checkpointing, barrier alignment, the differences between At‑Least‑Once and Exactly‑Once semantics, configuration options, incremental checkpointing, and the requirements for external sources and sinks to achieve end‑to‑end exactly‑once processing.

Apache Flinkbig datacheckpointing
0 likes · 15 min read
Understanding Fault Tolerance and Exactly-Once Semantics in Apache Flink
JD Tech
JD Tech
Mar 6, 2019 · Backend Development

Understanding Hystrix: Why It’s Needed and How to Use It for Dependency Isolation

This article explains why Hystrix is essential for large distributed systems, describes its dependency isolation mechanisms—including command, group, thread‑pool, and semaphore isolation—covers circuit‑breaker behavior, fallback strategies, and provides detailed Java code examples for configuration and usage.

Circuit BreakerHystrixJava
0 likes · 12 min read
Understanding Hystrix: Why It’s Needed and How to Use It for Dependency Isolation
Java Architect Essentials
Java Architect Essentials
Feb 25, 2019 · Backend Development

Service Isolation Design: Principles, Methods, and Best Practices

The article explains service isolation in system architecture, its origins, why it matters, two main isolation approaches (by service and by user), their advantages and drawbacks, and key considerations to ensure fault containment and improve overall system availability.

backend designfault tolerancemicroservices
0 likes · 7 min read
Service Isolation Design: Principles, Methods, and Best Practices
dbaplus Community
dbaplus Community
Feb 18, 2019 · Databases

How Do Fault‑Tolerant Transactions Work? Exploring Raft, KV Engines, and Concurrency Control

This article examines multiple fault‑tolerant transaction designs—RSM‑based KV, RSM‑based transactions, shared‑storage approaches, high‑availability KV layers, and single‑node engine extensions—comparing their replication strategies, lock handling, and performance trade‑offs while raising open questions about ordering and consistency.

KV storeRaftReplication
0 likes · 15 min read
How Do Fault‑Tolerant Transactions Work? Exploring Raft, KV Engines, and Concurrency Control
Architects Research Society
Architects Research Society
Jan 19, 2019 · Cloud Native

Three Common Microservices Integration Pitfalls and Their Mitigation Strategies

This article examines three frequent pitfalls encountered when integrating microservices—complex communication, asynchronous challenges, and distributed transaction difficulties—and proposes mitigation techniques such as rapid failure handling, workflow engines, timeout management, and compensation patterns to improve resilience and reduce system complexity.

Workflowcloud-nativefault tolerance
0 likes · 13 min read
Three Common Microservices Integration Pitfalls and Their Mitigation Strategies
Programmer DD
Programmer DD
Dec 21, 2018 · Backend Development

How Circuit Breakers Safeguard Distributed Systems from Cascading Failures

This article explains the concept of circuit breaking in distributed systems, outlines a four‑step implementation process with strategies for detecting unhealthy services, cutting off calls, probing recovery, and restoring normal operation, and shares best‑practice tips to minimize downtime and improve resilience.

Circuit Breakerdistributed systemsfault tolerance
0 likes · 10 min read
How Circuit Breakers Safeguard Distributed Systems from Cascading Failures
Architect's Tech Stack
Architect's Tech Stack
Dec 5, 2018 · Operations

Practical Fault‑Tolerance Practices in a Large‑Scale Activity Operations Platform

The article shares a comprehensive, experience‑driven guide on building fault‑tolerant systems—covering retry mechanisms, dynamic node removal, timeout settings, service degradation, decoupling, and business‑level safeguards—to enable a platform that scales from millions to billions of daily requests without relying on manual fire‑fighting.

Operationsfault toleranceretry mechanism
0 likes · 21 min read
Practical Fault‑Tolerance Practices in a Large‑Scale Activity Operations Platform
Programmer DD
Programmer DD
Oct 30, 2018 · Fundamentals

What Is Paxos? A Storytelling Guide to Distributed Consensus

This article uses a vivid allegorical story to introduce the Paxos algorithm, then explains its roles, two-phase protocol, fault assumptions, and why majority and multiple acceptors are essential for achieving reliable consensus in distributed systems.

AlgorithmDistributed ConsensusPaxos
0 likes · 10 min read
What Is Paxos? A Storytelling Guide to Distributed Consensus
UC Tech Team
UC Tech Team
Oct 23, 2018 · Operations

Understanding Faults and Fault Isolation Strategies in Distributed Systems

The article explains what constitutes a fault, introduces key metrics such as RPO and RTO, and describes various fault isolation principles, patterns, and practical examples—including dependency degradation, failover, dynamic adjustment, fast‑fail, caching, rate limiting, and resource isolation—to improve system reliability.

FailoverOperationsRPO
0 likes · 14 min read
Understanding Faults and Fault Isolation Strategies in Distributed Systems
Java Backend Technology
Java Backend Technology
Aug 18, 2018 · Backend Development

Why Service Isolation Is Essential for Fault‑Tolerant Backend Systems

The article explains the concept of service isolation, its origins in shipbuilding, why it’s crucial for reducing fault impact in software systems, practical approaches such as functional and user‑based isolation, their trade‑offs, and key design principles to ensure reliable, maintainable back‑end architectures.

backend architecturefault tolerancemicroservices
0 likes · 7 min read
Why Service Isolation Is Essential for Fault‑Tolerant Backend Systems
Meitu Technology
Meitu Technology
Aug 2, 2018 · Big Data

Spark Streaming vs Flink – Architecture, Scheduling & Fault Tolerance

This article compares Spark Streaming and Flink across runtime models, component roles, programming APIs, task scheduling, time semantics, dynamic Kafka partition detection, fault‑tolerance mechanisms, exactly‑once guarantees, and back‑pressure handling, providing code examples and practical insights for real‑time data processing.

Dynamic Partition DetectionFlinkKafka Integration
0 likes · 23 min read
Spark Streaming vs Flink – Architecture, Scheduling & Fault Tolerance
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Jul 20, 2018 · Backend Development

Mastering Service Discovery and Communication in Microservices

This article explains how microservices use service registries for discovery, registration, health checks, and deregistration, compares third‑party and self‑registration, explores server‑side and client‑side call mechanisms, discusses API gateways, synchronous vs asynchronous messaging, and outlines fault‑tolerance patterns such as timeouts, circuit breakers, and bulkheads.

API GatewayBackendService Discovery
0 likes · 21 min read
Mastering Service Discovery and Communication in Microservices
Architecture Digest
Architecture Digest
Jul 19, 2018 · Operations

How to Prevent System Failures: Suspect Third‑Party Services, Guard Consumers, and Strengthen Your Own Service

The article presents practical strategies for avoiding service failures by treating third‑party dependencies as unreliable, designing robust APIs for consumers, and applying solid engineering principles such as degradation plans, timeout settings, traffic control, and resource‑limiting techniques.

API Designfault tolerancereliability
0 likes · 16 min read
How to Prevent System Failures: Suspect Third‑Party Services, Guard Consumers, and Strengthen Your Own Service
ITPUB
ITPUB
Jun 6, 2018 · Cloud Native

How to Build a Cloud‑Native Microservices PaaS with Spring Cloud Netflix

This article explains how to construct a PaaS cloud platform using microservice architecture and Docker containers, detailing the roles of Spring Cloud Netflix components such as Zuul, Eureka, Hystrix, and Config Server, and covering gateway routing, service discovery, deployment, fault tolerance, and dynamic configuration.

Service DiscoverySpring Cloudfault tolerance
0 likes · 13 min read
How to Build a Cloud‑Native Microservices PaaS with Spring Cloud Netflix
Meituan Technology Team
Meituan Technology Team
May 31, 2018 · Operations

High‑Availability Practices for Account Services at Meituan/Dianping

Meituan/Dianping ensures its critical account service stays online by combining real‑time business monitoring, circuit‑breaker‑driven graceful degradation, and active‑active cross‑region deployment with isolated dependencies, versioned data sync, and automated cache updates, dramatically extending MTBF while cutting MTTR and latency.

High Availabilitydata synchronizationfault tolerance
0 likes · 13 min read
High‑Availability Practices for Account Services at Meituan/Dianping
Efficient Ops
Efficient Ops
May 27, 2018 · Operations

Mastering High Availability and High Concurrency: Principles and Practical Techniques

This article outlines guiding principles, high‑availability strategies, and high‑concurrency techniques—covering stateless design, resource isolation, quota management, monitoring, degradation, rollback, and scaling—to help engineers build resilient, scalable systems while balancing cost and performance.

Operationsfault tolerancehigh concurrency
0 likes · 21 min read
Mastering High Availability and High Concurrency: Principles and Practical Techniques
Efficient Ops
Efficient Ops
May 21, 2018 · Databases

Why Do Database Failures Happen and How to Prevent Them?

This article examines common hardware and network failures in data centers, analyzes real‑world outage cases, classifies fault domains, and presents comprehensive strategies for database fault handling—including logging, checkpointing, backup, replication, and high‑availability architectures—to improve reliability and reduce downtime.

Loggingbackupdatabase
0 likes · 22 min read
Why Do Database Failures Happen and How to Prevent Them?