Sentinel: Flow Control and Circuit Breaking for Microservice Stability

Guardian of Microservices

In distributed systems, a single failing interface can cause a cascade failure across the entire application chain. To withstand traffic spikes and sudden exceptions, stability measures are essential; Sentinel, a veteran in stability protection, achieves this through rate limiting and circuit breaking.

Flow Control

Rate limiting is one of Sentinel's core capabilities, aiming to protect the system under high concurrency or burst traffic. Its basic idea is to "control the amount, stabilize the scene" before a service is overwhelmed.

Sentinel offers flexible, fine‑grained limiting mechanisms that can protect resources from multiple dimensions such as QPS, thread count, call chain, and parameter level.

Common limiting types:

QPS limiting : controls access frequency by requests per second.

Thread count limiting : restricts the number of concurrent threads accessing a resource.

Associated limiting : limits a hot resource based on the health of its downstream services.

Chain limiting : applies limits at different entry points of a call chain.

Hot‑parameter limiting : limits the frequency of specific parameter values, such as a product ID during a flash‑sale.

Sentinel’s rate limiting is not merely a throttle; it acts as an intelligent valve that adjusts based on overall traffic, business importance, and hot parameters, enabling more accurate performance evaluation and robust production protection.

Degrade (Circuit Breaking)

Sentinel’s circuit‑breaking feature protects against downstream service failures or latency spikes, preventing avalanche effects in distributed systems.

The core idea is to temporarily cut off calls to an unhealthy service, giving the system a "cool‑down" period for self‑recovery, similar to a circuit breaker.

Sentinel supports three circuit‑breaking strategies:

Slow‑call ratio breaking : triggers when average response time exceeds a threshold and the proportion of slow calls stays above a set ratio.

Exception ratio breaking : triggers when the exception ratio exceeds a threshold after a minimum number of requests.

Exception count breaking : triggers when the total number of exceptions in a time window exceeds a threshold.

The circuit breaker operates as a state machine with three states:

Closed : normal operation, collecting metrics.

Open : all requests fail or are degraded.

Half‑open : after a configured break time, a trial period allows limited requests to test recovery.

When a request is broken, Sentinel can return default responses, invoke fallback logic, trigger alerts, or integrate with logging and tracing for further analysis.

Configuration Methods

Sentinel offers flexible configuration options:

Console configuration : graphical UI for adding, modifying, and deleting rules in real time.

Local rule configuration : hard‑coded rules within the project, suitable for testing or initialization.

Dynamic configuration via config centers (e.g., Nacos, Apollo): centralized rule management and push.

Rule persistence : rules survive restarts through local files or config‑center storage.

Underlying Mechanism

Sentinel’s core design follows the Chain‑of‑Responsibility pattern, using a ProcessorSlotChain where each request passes through a series of Slots.

StatisticSlot : sliding‑window statistics for QPS, RT, exception count, etc.

FlowSlot : core rate‑limiting logic.

DegradeSlot : decides whether to break based on exception rate or response time.

SystemSlot : monitors system resources such as load and memory.

The sliding‑window mechanism is implemented with LeapArray , dividing the monitoring period into buckets that record metrics, providing real‑time, high‑precision statistics compared to simple counters, leaky‑bucket, or token‑bucket algorithms.

Practical Experience: Scenario Matching

For sudden traffic spikes, use Sentinel’s warm‑up mode with QPS limits and a reasonable warm‑up period to avoid cold‑start overload.

When response times fluctuate, enable the slow‑call ratio breaking strategy with appropriate thresholds (e.g., 800 ms and 50% slow‑call ratio).

During overall load increase, configure System Rules (CPU, load, thread count, QPS) to automatically degrade traffic and protect core services.

For hot‑parameter attacks (e.g., frequent keyword searches), apply hotspot parameter limiting with per‑value thresholds and optional whitelist exceptions.

Combining these strategies with realistic load testing validates the effectiveness of rate limiting and circuit breaking, enhancing system resilience.

Performance Testing and Fault Testing

In performance testing, Sentinel not only limits traffic but also simulates real‑world burst scenarios through QPS limiting, concurrency control, and warm‑up modes, helping identify cold‑start issues and capacity limits.

In fault testing, Sentinel acts as a controllable sandbox, using degrade rules and system rules to emulate service timeouts, latency spikes, and resource exhaustion, providing fine‑grained fault injection without destructive tools.