Microservice Traffic Defense: 5-Layer Rate Limiting & Risk Control for Production Systems

Many teams implement only global rate limiting, yet still suffer from crawlers, API brushing, and malicious attacks that bypass simple thresholds. This article presents a complete five-layer defense system validated in production.

Why Global Rate Limiting Fails

Three fatal problems emerge with naive global QPS limits:

Malicious traffic monopolizes thresholds: A few scripts consume the global quota, blocking legitimate users ("bad actors pass, good users blocked").

Distributed crawlers evade detection: Multi-IP rotation keeps per-IP rates below the global threshold while aggregate traffic overwhelms the service.

No tiered protection causes cascade failure: Core trading APIs share limits with non-critical queries; traffic spikes on low-priority endpoints starve critical paths, triggering avalanches during peak promotions.

Five-Layer Traffic Protection Architecture

A mature defense operates from outside in, each layer addressing a distinct threat:

Layer 1 — Nginx / Edge Gateway: Blocks DDoS, malicious IPs, anomalous traffic via IP blacklist and high-frequency bans.

Layer 2 — Spring Cloud Gateway: Provides global traffic shaping and peak shaving through cluster-wide rate limiting and unified circuit breaking.

Layer 3 — Interface Dimension: Delivers differentiated protection for core APIs with independent QPS and tiered rate limiting.

Layer 4 — User / IP Dimension: Prevents single-user or single-IP abuse via per-IP and per-account rate limiting.

Layer 5 — Business Risk Control: Intercepts malicious business operations using anomaly behavior checks and CAPTCHA challenges.

Four Rate Limiting Algorithms: Production Comparison

Fixed Window: Simplest code, lowest overhead. Weakness: boundary burst-through at window edges. Production fit: admin/internal non-critical endpoints.

Sliding Window: Precise, smooth traffic, no spikes. Weakness: moderate memory cost. Production fit: core query APIs, general user-facing endpoints.

Leaky Bucket: Enforces constant rate, strong peak shaving. Weakness: cannot absorb legitimate bursts. Production fit: MQ consumers, async task processing.

Token Bucket: Allows bursts, elastic throughput. Weakness: parameters need business tuning. Production fit: C-end core transaction APIs (top choice) .

Selection verdict: C-end core interfaces → Token Bucket; async consumption → Leaky Bucket; core queries → Sliding Window.

Fine-Grained IP & User Rate Limiting (Redis + Sliding Window)

Global limits are blunt; effective defense requires per-IP and per-user granularity. The author's go-to production pattern uses Redis sorted sets with a Lua script for atomicity, eliminating race conditions in distributed clusters.

4.1 Single-IP Sliding Window (Anti-Crawler Core)

Logic: 60-second window, max 60 requests per IP. Expired entries are purged atomically via Lua.

@Component
public class IpLimitFilter {
    @Resource
    private StringRedisTemplate redisTemplate;
    private static final String LIMIT_KEY = "limit:ip:";
    private static final int WINDOW_SECOND = 60;
    private static final int MAX_COUNT = 60;

    public boolean isAllow(String ip) {
        String key = LIMIT_KEY + ip;
        long now = System.currentTimeMillis();
        String luaScript = "local now = tonumber(ARGV[1]) " +
                "local window = tonumber(ARGV[2]) " +
                "redis.call('ZREMRANGEBYSCORE', KEYS[1], 0, now - window) " +
                "local count = redis.call('ZCARD', KEYS[1]) " +
                "if count < tonumber(ARGV[3]) then " +
                "redis.call('ZADD', KEYS[1], now, now..math.random()) " +
                "redis.call('EXPIRE', KEYS[1], 60) " +
                "return 1 " +
                "end " +
                "return 0";
        DefaultRedisScript<Long> script = new DefaultRedisScript<>(luaScript, Long.class);
        Long result = redisTemplate.execute(script, Collections.singletonList(key),
                String.valueOf(now), String.valueOf(WINDOW_SECOND * 1000), String.valueOf(MAX_COUNT));
        return Long.valueOf(1).equals(result);
    }
}

4.2 Per-Account Rate Limiting Supplement

IP limits alone are insufficient; attackers rotate IPs. For authenticated endpoints, add a userId dimension with identical logic (key = limit:user:{userId}). This dual check stops single-account high-frequency abuse, API enumeration, and coupon-farming scripts.

Interface Tiering: Protecting Core Paths During Promotions

Promotion failures often stem not from insufficient hardware but from missing traffic tiering. Non-core traffic surges crowd out critical paths. The author standardizes a three-tier model:

P0 Core Chain (highest priority): Order, payment, refund, inventory deduction, account debit. Highest thresholds; no degradation or limiting during peaks; transaction closure guaranteed.

P1 Regular Business (medium): Product lists, detail queries, personal order history. Moderate thresholds; slight limiting acceptable without UX impact.

P2 Non-Core (lowest): Log stats, reports, admin ops, user footprints. Minimal thresholds; manually degraded or fully discarded during peaks.

Core principle: Do not aim for all interfaces available; guarantee only core business availability.

Dynamic Blacklist System: Stopping Persistent Attacks

Rate limiting is a "soft" throttle; it cannot stop determined attackers. A dynamic blacklist provides hard blocks.

General Ban Rules

Trigger limit 3+ times in 1 minute → suspicious → temp ban 5 minutes .

High-frequency 404s, illegal paths, malformed params → probe/enumeration attack → ban 30 minutes .

Sustained high-volume abuse, malicious endpoint targeting → permanent blacklist , manual review required.

Atomic Blacklist Check Lua Script

Ensures check-count-ban executes atomically, preventing bypass under concurrency.

-- KEYS[1]: blacklist key  KEYS[2]: request counter key
-- ARGV[1]: max allowed count  ARGV[2]: ban duration (seconds)
if redis.call('get', KEYS[1]) then
    -- already blacklisted, reject immediately
    return 0
end
-- if over limit, add to temporary blacklist
if tonumber(redis.call('get', KEYS[2]) or 0) > tonumber(ARGV[1]) then
    redis.call('set', KEYS[1], 1, ARGV[2])
    redis.call('del', KEYS[2])
    return 0
end
return 1

Real Incident Review: Crawler Scraped All Product Data

Symptoms

Off-peak early morning, unattended: CPU/memory saturated, logs flooded with product list/detail requests, next-day user pages extremely slow, some APIs timing out.

Root Cause Analysis

Access logs showed many unknown IPs polling product list/detail endpoints at uniform rates — classic crawler pattern.

Only global QPS limit existed; no per-IP cap. Crawler distributed requests across IPs, perfectly evading the global threshold.

High-frequency queries hammered cache and database, spiking pressure and dragging down the entire microservice cluster.

Fixes Deployed

Enabled per-IP sliding window rate limiting (max requests/minute).

Activated dynamic blacklist to auto-ban high-frequency crawler IPs.

Staticized product detail pages to reduce live DB queries.

Auto-degraded non-core query endpoints at night to cut resource waste.

Result: Crawler traffic dropped to zero; nighttime server load fell 90%.

Architect's Six Core Takeaways

Global rate limiting is only a last-resort safety net; real protection lives at IP/user granularity.

Crawlers and scripts are distributed low-frequency; blunt one-size-fits-all limits are useless — precision interception is mandatory.

Every production system must classify interfaces; core business gets priority, non-core is degradable and disposable.

For C-end high concurrency, prefer Token Bucket to balance burst absorption and stability.

Rate limiting handles volume; blacklists handle malice — both are required together.

Complete protection loop: Gateway global limit → Interface tiering → User/IP rate limiting → Business behavior risk control → Blacklist final ban .

Many seemingly complex outages boil down to missing foundational defenses. Deploying this traffic risk control system raises service stability by at least one level.