Tagged articles
degradation
40 articles · Page 1 of 1
This article details a production-grade high-availability framework for vLLM-based LLM inference services on Kubernetes, covering layered health checks, multi-level model warmup, automated failover, degradation strategies, metrics-driven alerting, and troubleshooting patterns derived from real incidents.
FailoverGPUHigh Availability
0 likes · 57 min read

This article explains why retry alone is insufficient for fault tolerance, detailing how to classify failures via fault models, apply appropriate mechanisms like timeouts and circuit breakers, design recovery paths targeting state convergence, define RTO/RPO for disaster recovery, and validate all assumptions through fault injection drills.
Circuit BreakerIdempotencyRPO
0 likes · 23 min read

This article analyzes three real production incidents from e-commerce promotions to expose critical flaws in circuit breaker and degradation architectures, detailing the dangers of half-open state probing, configuration hot-update inconsistency, and unbounded business degradation, then presents a three-layer protection model with custom recovery strategies and governance checklists.
Circuit BreakerHigh Availabilityarchitecture governance
0 likes · 23 min read

This article outlines a production-grade AI runtime architecture that separates processing state from business conclusions, enforces bounded model execution with retry budgets and circuit breakers, ensures idempotent external actions, defines explicit degradation paths, structures human-in-the-loop workflows, extends observability to judgment quality, and validates rollback and failure injection before launch.
AI productionCircuit BreakerIdempotency
0 likes · 27 min read

This article details a complete microservices high-availability system using Sentinel, covering dual-layer rate limiting, circuit breaking with three states, active degradation strategies, and fallback methods to prevent cascade failures during traffic spikes and downstream failures.
Cascade FailureHigh AvailabilitySentinel
0 likes · 16 min read


liandkAug 10, 2026 · Backend Development
To keep microservice systems stable, the article explains why fault‑tolerance is essential, defines service avalanches, and details four core safeguards—rate limiting, circuit breaking, degradation, and debounce—covering their principles, typical use cases, deployment layers, and how they work together to prevent cascading failures.
Circuit Breakerdebouncedegradation
0 likes · 8 min read

The article explains how to prevent system crashes during massive traffic spikes by applying three core techniques—rate limiting, circuit breaking, and graceful degradation—detailing algorithm choices, state machines, and practical implementation steps for high‑concurrency back‑end services.
BackendCircuit Breakerdegradation
0 likes · 6 min read

The article explains that high availability means designing systems to keep serving after failures through redundancy, fault detection, failover, degradation, and clear RTO/RPO goals, rather than trying to avoid any downtime altogether.
FailoverHigh AvailabilityRPO
0 likes · 13 min read

The article explains how microservice decomposition introduces challenges such as locating services, routing external requests, balancing load, isolating failures, limiting traffic, and degrading non‑essential features, and it details the six core mechanisms—service discovery, API gateway, load balancing, circuit breaking, rate limiting, and degradation—that together form a complete service‑governance solution.
API gatewayCircuit Breakerdegradation
0 likes · 12 min read

This article provides a comprehensive overview of Alibaba’s open‑source Sentinel, detailing its core components, flow‑control strategies—including threshold, multi‑dimensional, hotspot, and adaptive limiting—along with circuit‑breaking and degradation mechanisms, and includes practical Java code examples for integration with Spring Cloud and Dubbo.
Flow ControlJavaSentinel
0 likes · 4 min read

This article presents a comprehensive design of Bilibili's comment service architecture, detailing the transition from TiDB to a multi‑level storage system based on Taishan KV, the data models, consistency mechanisms, retry and versioning strategies, and a hedging‑based degradation policy to ensure high availability under heavy traffic.
Taishan KVTiDBcomment system
0 likes · 13 min read

The article explains fault isolation as a key technique for improving distributed system availability, detailing multiple isolation levels—from data‑center to user‑level—and complementary strategies such as circuit breakers, timeouts, fast‑fail, load balancing, caching, and degradation switches.
Circuit BreakerFault Isolationdegradation
0 likes · 10 min read

The article explains how interface‑level faults—where the system stays up but business performance degrades—can be mitigated through four core techniques (degradation, circuit breaking, rate limiting, and queuing), detailing their principles, implementation patterns, and practical trade‑offs for backend services.
BackendCircuit Breakerdegradation
0 likes · 20 min read

The article explains interface‑level failures in business systems and presents four mitigation strategies—degradation, circuit breaking, rate limiting, and queuing—detailing their principles, implementation methods, and algorithmic choices such as fixed and sliding windows, token bucket and leaky bucket.
Backendcircuit breakingdegradation
0 likes · 18 min read

The article explains what interface‑level failures are, why they occur due to internal bugs or external overload, and presents four practical mitigation techniques—degradation, circuit breaking, rate limiting, and queuing—detailing their principles, implementation options, and trade‑offs for reliable system operation.
Circuit BreakerQueuedegradation
0 likes · 16 min read

This article outlines comprehensive best‑practice guidelines for designing robust, secure, and maintainable backend services—covering API path conventions, request handling, parameter design, error codes, dependency management, monitoring, degradation strategies, legacy service handling, encryption, access control, and tamper‑proof mechanisms, with practical code examples.
API DesignBackenddegradation
0 likes · 18 min read

This article explores essential service design considerations beyond standard guidelines, covering API path structuring, request handling, parameter design, security measures, monitoring, degradation strategies, and code examples to help build flexible, secure, and maintainable backend services.
API Designbackend developmentdegradation
0 likes · 19 min read

The article explains how traffic governance—including circuit breaking, isolation, retries, degradation, timeout handling, and rate limiting—maintains the three‑high goals of high performance, high availability, and easy scalability in microservice architectures, using practical examples and formulas.
RetryTimeoutcircuit-breaker
0 likes · 29 min read

This article explains how traffic governance—including circuit breaking, isolation, retry mechanisms, degradation, timeout control, and rate limiting—helps microservice systems achieve the three‑high goals of high performance, high availability, and easy scalability, using concrete formulas, algorithms, and practical examples.
High AvailabilityRetryTimeout
0 likes · 29 min read

This article explains how to achieve high availability in microservice architectures by applying flow governance techniques such as circuit breaking, isolation, retry policies, degradation, timeout management, and rate limiting, while detailing key metrics like MTBF and MTTR and providing practical implementation guidance.
Flow ControlHigh AvailabilityRetry
0 likes · 30 min read

This article compares three core system‑level high‑availability strategies—circuit breaking, degradation, and rate limiting—explaining their definitions, typical scenarios, design principles, and technical value so leaders can assess ROI and choose the right protection mechanism.
Circuit BreakerHigh AvailabilityOperations
0 likes · 7 min read

This article explains the concept of high concurrency, its challenges such as performance degradation and resource contention, and presents comprehensive solutions including various caching mechanisms, multiple rate‑limiting algorithms, and degradation and circuit‑breaker strategies to ensure system stability under heavy load.
Circuit Breakerdegradationhigh concurrency
0 likes · 43 min read

This article explains how to reduce the impact of failures in distributed systems by simplifying service links, applying circuit‑breaker mechanisms, implementing graceful degradation, performing core‑link isolation, and, as a last resort, switching to a minimal MVP version to keep essential functionality alive.
Circuit BreakerHigh AvailabilityOperations
0 likes · 11 min read

This article explains the architectural mindset behind high concurrency, defines key performance metrics such as TPS, RPS, RT and VU, analyzes test results, and presents practical techniques like rate limiting, frequency limiting, degradation and caching to balance performance, availability, and resource usage.
Cachingdegradationhigh concurrency
0 likes · 27 min read

This article explains the concepts of rate limiting, degradation, and circuit breaking in microservice architectures, illustrating passive and active throttling strategies, practical examples of async conversion, various degradation techniques, and circuit‑breaker mechanisms with real‑world tools like Sentinel and Hystrix.
circuit breakingdegradationmicroservices
0 likes · 11 min read

To deliver a flawless live broadcast of the 2021 League of Legends S11 World Championship to over 100 million viewers, Bilibili mobilized hundreds of engineers for four months, establishing strict standards, modeling dozens of user scenarios, estimating traffic, conducting layered stress and chaos tests, implementing automated and manual degradation, detailed SOPs, rate‑limiting safeguards, and on‑site monitoring, which together ensured system stability throughout the event.
degradationrate limitingstress testing
0 likes · 14 min read

The article explains how to properly deal with message‑queue (MQ) outages by first encapsulating MQ operations, then applying degradation tactics such as persisting failed messages to a database, disk, or log, and finally implementing scheduled or manual message‑replay mechanisms while emphasizing monitoring and fallback logic.
BackendMQMessage Queue
0 likes · 5 min read

This guide outlines essential incident‑response principles—prioritizing business restoration and timely escalation—while detailing practical methods such as restart, isolation, and degradation, and explains how to organize response teams and conduct thorough post‑incident reviews.
Incident Managementdegradationfault handling
0 likes · 11 min read

This article explains the role of caching, service degradation, and flow control in high‑concurrency systems, introduces common rate‑limiting algorithms such as counters, leaky bucket and token bucket, and provides practical Java and Nginx implementations with code examples.
CachingLeaky Bucketdegradation
0 likes · 14 min read


21CTOFeb 26, 2020 · Backend Development
This article explains how caching, degradation, and various rate‑limiting strategies—including semaphore‑based concurrency control, token‑bucket algorithms, Guava RateLimiter, custom annotations, Redis interceptors, and Nginx modules—protect high‑concurrency distributed systems, with practical Java code samples and configuration snippets.
Cachingdegradationdistributed systems
0 likes · 19 min read

This article outlines comprehensive internal resource governance techniques—including degradation, circuit breaking, isolation, async conversion, thread‑pool management, JVM and hardware metric monitoring, and daily operational practices—to enhance system stability and high availability in large‑scale backend services.
Circuit Breakerbackend operationsdegradation
0 likes · 10 min read

The article details how 7FRESH's R&D, testing, network operations, and product teams coordinated system capacity expansion, degradation mechanisms, emergency response procedures, extensive performance testing, and 24/7 monitoring to ensure stable and scalable service during the high‑traffic 11.11 shopping event.
Operationscapacity planningdegradation
0 likes · 10 min read

At QCon Beijing 2019, senior architect Wang Junxiang shared Tong‑Cheng Yilong’s end‑to‑end degradation‑plan architecture, covering system design, data collection, metric computation, resource recovery, link‑level pre‑plan management, fault diagnosis, strategy extensibility, and high‑availability platform construction, offering practical insights for complex distributed systems.
High Availabilitydegradationdistributed systems
0 likes · 4 min read

This article outlines eight practical high‑availability techniques—multiple replicas, isolation, rate limiting, circuit breaking, degradation, gray releases with rollback, comprehensive monitoring, and proactive log alerting—to help engineers design systems that are both efficient and reliable under heavy load.
Circuit BreakerGray ReleaseHigh Availability
0 likes · 7 min read

The article explains why interface-level failures occur, distinguishes internal and external causes, and presents four practical strategies—degradation, circuit breaking, rate limiting, and queuing—to keep core services running and protect most users under heavy load.
Circuit BreakerQueuebackend reliability
0 likes · 6 min read

This article explains the concept of service degradation, why it is needed to maximize limited resources during traffic spikes, outlines common degradation strategies, and provides practical steps and code examples for ranking, sequencing, and implementing degradation in both front‑end and back‑end systems.
High AvailabilityOperationsdegradation
0 likes · 11 min read

The article outlines Meituan Finance’s front‑end availability challenges in its million‑order payment service and presents a disciplined, end‑to‑end approach—standardized release processes, simple fallback designs, automated testing, robust monitoring, and regular fault‑drill simulations—to ensure stable user experiences across diverse client environments.
Best PracticesMeituanavailability
0 likes · 17 min read

This article examines how Datagateway, a centralized data access layer, tackles high‑traffic challenges through layered architecture, distributed caching strategies, thundering‑herd mitigation, hot‑key handling, disaster‑recovery mechanisms, and degradation‑circuit‑breaker techniques to maintain system stability and performance.
BackendCachingdegradation
0 likes · 15 min read

This article presents a comprehensive redesign of Baidu Waimai’s client‑side activity module, detailing background challenges, design goals, functional and performance specifications, trade‑off analyses of three architectural alternatives, and the chosen parallel HTTP‑request solution with monitoring, degradation, and phased rollout plans.
Backendarchitecturedegradation
0 likes · 8 min read

The article explains how functional separation and degradation techniques—distinguishing core from non‑core services, isolating them logically and physically, and implementing manual or automatic fallback mechanisms—help maintain high availability in distributed systems during traffic spikes or component failures.
Operationsdegradationfunctional separation
0 likes · 3 min read
