Sentinel Deep Dive: Flow Control, Circuit Breaking & Rate Limiting

1. Introduction

This article introduces Alibaba Sentinel, a mature open‑source flow‑control and circuit‑breaking project that has been battle‑tested during nearly ten years of Double‑11 shopping festivals.

2. What is Sentinel?

Sentinel, literally "sentinel", is similar to Redis Sentinel for master‑slave monitoring, but in microservices it acts as a traffic guard. It protects services from overload by controlling traffic, circuit breaking, and system load.

Key features include:

Rich application scenarios : Handles core scenarios of Alibaba's Double‑11 traffic, such as flash sales, message throttling, cluster traffic control, and real‑time circuit breaking.

Real‑time monitoring : Provides per‑machine second‑level metrics and cluster‑wide summaries.

Broad open‑source ecosystem : Integrates with Spring Cloud, Apache Dubbo, gRPC, Quarkus, and offers native implementations for Java, Go, C++.

Extensible SPI mechanism : Allows custom rule management and dynamic data source adaptation.

3. Sentinel vs. Hystrix

Sentinel outperforms Hystrix in many aspects and is recommended as a replacement.

4. Choosing Sentinel version

When using Spring Cloud Alibaba, the Sentinel version should match the Spring Cloud version. For example, with spring-cloud-alibaba-dependencies 2.2.1.RELEASE, use Sentinel 1.7.1. Ensure JDK >= 1.8.

java -Dserver.port=8080 -Dcsp.sentinel.dashboard.server=localhost:8080 -Dproject.name=sentinel-dashboard -jar sentinel-dashboard-1.7.1.jar

Parameters: -Dserver.port: Port (default 8080) -Dproject.name: Service name -Dcsp.sentinel.dashboard.server: Dashboard address

5. Installing the Sentinel dashboard

Download the appropriate JAR from the Sentinel GitHub releases and run it. Default credentials are sentinel/sentinel. You can also build from source with mvn clean package.

6. Flow control configuration

Sentinel protects services by limiting QPS or concurrent threads. A flow rule consists of:

resource : Target resource name

count : Threshold value

grade : 1 for QPS, 0 for concurrency

limitApp : Caller source (default default)

strategy : 0 (self), 1 (related), 2 (chain)

controlBehavior : 0 (fast fail), 1 (warm‑up), 2 (queue)

Three flow‑control effects:

Fast Fail

Requests exceeding the threshold are immediately rejected with a FlowException.

Warm‑up

Gradually ramps up traffic when the system has been idle, using a token‑bucket algorithm. The coldFactor defaults to 3.

Queue Waiting

Requests are queued and processed at a uniform rate; if waiting exceeds the configured timeout, the request is rejected.

7. Degradation (circuit breaking) rules

Sentinel supports three degradation strategies:

Average response time (DEGRADE_GRADE_RT) : If the average RT exceeds the threshold for a period, the method is circuit‑broken.

Exception ratio (DEGRADE_GRADE_EXCEPTION_RATIO) : If the exception ratio exceeds the configured value, the method is circuit‑broken.

Exception count (DEGRADE_GRADE_EXCEPTION_COUNT) : If the number of exceptions in a minute exceeds the threshold, the method is circuit‑broken.

8. Hotspot parameter limiting

Hotspot limiting controls frequently accessed parameters (e.g., product IDs in flash‑sale scenarios) using an LRU cache and token‑bucket algorithm. It only works for QPS, not concurrency.

9. System adaptive limiting

System rules protect the whole application based on metrics such as load, CPU usage, average RT, concurrent threads, and entry QPS. Five threshold types are supported: Load, CPU usage, average RT, concurrent threads, and entry QPS.

10. Customizing block handler messages

Use @SentinelResource with blockHandler (or blockHandlerClass) to define custom responses when a request is blocked.

@SentinelResource(value = "OrderQuery", blockHandler = "handlerQuery")
public String query(String p1, String p2) { ... }
public String handlerQuery(String p1, String p2, BlockException ex) { return "Query blocked"; }

11. Fallback for exceptions

Define fallback (or fallbackClass) in @SentinelResource to handle business exceptions. defaultFallback provides a global fallback.

@SentinelResource(value = "createOrder", fallback = "fallbackCreate", blockHandler = "blockCreate")
public String createOrder() { int i = 1/0; return "ok"; }
public String fallbackCreate(Throwable e) { return "fallback"; }
public String blockCreate(BlockException e) { return "blocked"; }

12. Black‑white list control

Sentinel can allow or deny requests based on origin using AuthorityRule. Implement RequestOriginParser to extract the origin (e.g., IP address).

public class IpOriginParser implements RequestOriginParser {
    @Override
    public String parseOrigin(HttpServletRequest request) {
        return request.getRemoteAddr();
    }
}

13. Rule persistence with Nacos (Push mode)

Sentinel rules can be persisted in Nacos. Add sentinel-datasource-nacos dependency and configure spring.cloud.sentinel.datasource for flow and degrade rules.

spring:
  cloud:
    sentinel:
      datasource:
        ds-flow:
          nacos:
            server-addr: 127.0.0.1:8848
            dataId: ${spring.application.name}-flow
            groupId: DEFAULT_GROUP
            data-type: json
            rule-type: flow
        ds-degrade:
          nacos:
            server-addr: 127.0.0.1:8848
            dataId: ${spring.application.name}-degrade
            groupId: DEFAULT_GROUP
            data-type: json
            rule-type: degrade

Push mode automatically syncs Nacos changes to the dashboard. To sync dashboard changes back to Nacos, modify Sentinel’s dashboard source code (replace FlowControllerV1 with the V2 implementation and adjust Nacos provider/publisher beans).

14. Cluster flow control (Embedded mode)

Cluster flow control uses a token server to enforce global QPS limits across multiple instances. In embedded mode, each instance runs a token client and one instance acts as the token server.

Configure token server address in application.yml and enable cluster mode in Sentinel rules.

sentinel:
  transport:
    port: 8719
  flow:
    cluster-mode: true
    server-addr: 127.0.0.1:8719

15. Additional topics

The article also mentions gateway rate limiting, OpenFeign integration, and provides extensive code examples and images illustrating each configuration step.