Principles and Practices of Rate Limiting Across SLB, Nginx, Spring Cloud Gateway, and Microservice Layers

In the introduction, the author emphasizes that rate‑limiting should be applied as close to the traffic source as possible and should match the team's skill set.

SLB node : Almost all external traffic (including DDoS, crawlers, and normal requests) passes through the SLB ingress. The primary goal is to block non‑business traffic before it reaches the application layer. Typical controls include IP‑based filtering, DDoS mitigation, and basic request throttling.

Nginx node : After SLB, traffic is handed to Nginx where more granular limits can be configured in code (e.g., connection concurrency, per‑IP request caps, crawler throttling). Nginx allows scriptable, version‑controlled configurations, making recovery fast.

Spring Cloud Gateway node : This layer handles dynamic Java‑based traffic. Rate‑limiting becomes more sophisticated, supporting normal flow limits, burst scenarios (e.g., flash sales), and CC‑attack mitigation with signature verification. Custom KeyResolver implementations enable limits by user ID, IP, tenant, etc.

Java microservice node : Individual services have their own capacity measured by load‑testing tools like JMeter. Limits may be applied per service based on sensitivity (e.g., billing). Techniques include queue‑based throttling and dedicated thread pools to isolate heavy‑load services.

The article also discusses isolation strategies (using Hystrix, Sentinel, Guava) to prevent a slow service from dragging down others, and introduces circuit‑breaker concepts for handling downstream timeouts.

Overall, the piece provides a layered approach to traffic control, recommending that each layer enforce appropriate limits, use code‑driven configurations where possible, and employ isolation and circuit‑breaking to maintain system resilience.