Mastering Rate Limiting: From SLB to Spring Cloud Gateway and Microservices

Principle of Rate Limiting

Rate limiting should be applied as close to the traffic source as possible, using the skills available in the team.

SLB Node Characteristics and Controls

All external traffic enters through the SLB (income) node, including business and non‑business traffic, DDoS, normal requests, crawlers, etc.

Intercept DDoS attacks.

Block unsafe traffic such as SQL injection, XSS.

Typical rate‑limiting measures at the SLB level include connection concurrency limits, per‑IP request limits, and crawler throttling. Because SLB configurations are UI‑based and not easily stored as code, teams often shift these rules to Nginx or Kong where they can be version‑controlled.

Nginx Node Characteristics and Rate Limiting

After SLB filtering, Nginx handles remaining traffic, which may still contain crawlers (sometimes needed for SEO) and advanced attacks.

Crawler throttling, concurrency control, or redirection to a honeypot.

Per‑IP request limiting.

Spring Cloud Gateway Node Characteristics

Gateway receives only dynamic Java‑based traffic; static H5 resources have already been redirected elsewhere. The traffic load is lower than Nginx, so stricter limits are required.

General rate limiting.

Burst traffic control (e.g., flash‑sale spikes).

CC attack mitigation and signature verification.

Dynamic routing can carry individual rate‑limit configurations.

Isolation and Circuit Breaking

Because a gateway forwards requests to many microservices, a single slow service must not affect others. Isolation can be achieved with tools such as Hystrix, Sentinel, or Guava.

Differences Between Nginx and Gateway Rate Limiting

Nginx thresholds are higher because it serves a broader range of traffic.

Gateway limits are more flexible; custom KeyResolver can define dimensions like user ID, IP, tenant, etc.

Microservice Node Rate Limiting

Each microservice’s capacity can be measured with load‑testing tools (e.g., JMeter) and used as a baseline for limits.

Apply fine‑grained limits for sensitive services (e.g., billing).

Use queue‑based throttling to fix consumer concurrency.

Thread‑Pool Isolation and Circuit Breaker Strategy

High‑traffic services should separate inbound request handling into dedicated thread pools and also isolate outbound calls to other services. When downstream services become slow or timeout, circuit breaking should return a predefined fallback response instead of propagating the delay.

Overall, a layered rate‑limiting strategy—from SLB to Nginx, to Gateway, to individual microservices—combined with isolation, circuit breaking, and proper thread‑pool design, helps maintain system stability under heavy or malicious traffic.