Nginx Rate Limiting Deep Dive: Leaky Bucket, Burst Config & Gateway vs Business Layer
This article analyzes Nginx rate limiting using limit_req and limit_conn, explaining the leaky bucket algorithm, burst and nodelay parameters, three production incidents, multi-layer defense configuration, cluster rate limiting strategies, key trust chains, monitoring, and interview questions.
Three Incidents, Three Lessons
The article opens with three real-world incidents that illustrate core pitfalls:
Incident 1: Normal users hit 503. An order API configured with limit_req rate=10r/s but no burst. The leaky bucket semantics mean 10r/s equals a constant rate of one request every 100ms. Users clicking twice within 100ms got the second request rejected. Rate is a smooth rate, not a burst ceiling; omitting burst punishes all normal users.
Incident 2: Database pressure unchanged after rate limiting. Four Nginx nodes each configured with rate=100r/s to protect the order database. Load testing showed 400r/s total throughput because each node's limit_req counter lives in its own worker shared memory — nodes are completely independent. Cluster rate limiting requires either deriving per-node rates from the global target or using Redis/OpenResty for a truly global counter.
Incident 3: SMS interface abused for 20,000 messages overnight. The SMS endpoint used limit_req rate=1r/m with key $http_x_forwarded_for. Because a CDN sat in front, the team assumed the real IP was in X-Forwarded-For. Attackers forged X-Forwarded-For headers, each forged header becoming a "new user." X-Forwarded-For is a client-controllable header; rate-limit keys must come from trusted sources only. For direct connections use binary_remote_addr; with CDN/LB use the real_ip module to strip trusted proxy layers first.
Technical Analysis: Leaky Bucket Precise Semantics
1.1 The rate, burst, nodelay Trio
limit_reqimplements a standard leaky bucket: requests pour in like water, the bucket drains at the constant rate. rate is the drain speed, burst is the bucket depth. Three combinations yield three distinct behaviors:
Only rate — bucket depth zero; any request exceeding the constant rate is immediately rejected. Suitable only for "strictly uniform call" scenarios (Incident 1).
rate + burst — allows up to burst requests to accumulate, but excess over rate must queue in the bucket waiting for release , adding visible latency.
rate + burst + nodelay — accumulated allowance is processed immediately without queuing , though bucket occupancy is still accounted; once allowance is exhausted, requests are rejected. This is the correct choice for most APIs: tolerate bursts, don't add pointless latency.
A frequent misunderstanding: rate=10r/s is not a "10 requests per second" sliding window counter, but a constant rate of one request every 100ms. A window counter would allow 10 requests in the first 100ms then idle for 900ms; the leaky bucket shapes traffic extremely smoothly. This difference explains why burst requests are rejected or queued even when the average rate is under the limit. Similarly, rate=10r/m means one request every 6 seconds, not "first 10 requests free each minute."
1.2 limit_req vs limit_conn: Complementary Gates
The two modules are often confused but have clear division of labor:
limit_req controls rate — how many requests per second; defends against "flooding."
limit_conn controls concurrency — maximum simultaneous connections per key; defends against "occupation."
Slow attacks (Slowloris, slow POST) keep request rate low but hold hundreds of connections; limit_req sees nothing, limit_conn cuts them off. Conversely, a sudden spike with low concurrency but huge rate is invisible to limit_conn but caught by limit_req. Production configurations must pair both: rate limiting handles "how fast they come," connection limiting handles "how many hang on." Missing either leaves a bypass dimension.
1.3 rate Syntax and Memory Accounting
Zone memory must be calculated: each limit_req_zone key state consumes ~64 bytes (IP, timestamp, water level). A 10m zone theoretically holds ~160,000 IP states. limit_conn_zone nodes are smaller; 10m can sustain hundreds of thousands of concurrent connections. When a zone fills, new keys cannot allocate state and are treated as "over limit" (default reject). Public entry zones should be generous (20m–64m); internal admin ports can be smaller. rate supports r/s and r/m units; low rates must use r/m for clarity — rate=30r/m is far clearer than rate=0.5r/s and avoids accidental 5r/s magnitude errors.
1.4 Key Trust Chain: The Foundation of Rate Limiting
All rate limiting rests on "key correctness," which depends on the traffic path:
Client connects directly to Nginx — binary_remote_addr is the true source (binary used purely to save memory: 4-byte fixed length).
CDN or Layer-4 LB in front — remote_addr becomes the proxy IP, collapsing all users into one key (either mass false positives or attacker exhausts quota with one IP). The real_ip module is mandatory: set_real_ip_from declares trusted proxy CIDRs, real_ip_header X-Forwarded-For peels from the trusted layer inward; the resulting $remote_addr is the real IP. Incident 3's mistake was using $http_x_forwarded_for directly — it's a request header the client can forge. If any link in the trust chain allows client injection, the entire defense collapses.
1.5 Three-Dimensional Defense: Three-Layer Zone Division
Single-point rate limiting cannot stop multi-dimensional abuse. Production requires three layers:
Global layer — coarse IP granularity (e.g., 50r/s + burst 100); blocks scripts, crawlers, obvious abuse; threshold intentionally loose.
Site layer — split by business domain (search 20r/s, order 10r/s); protects specific service capacity boundaries.
Interface layer — high-risk endpoints with finer keys and stricter limits (login by "IP+account" 1r/m, captcha 30r/m); prevents credential stuffing and scraping.
Each layer uses independent zones and parameters; a request hitting multiple layers is rejected if any layer rejects. The essence: "thresholds map one-to-one to protected objects" — global protects machines, site protects services, interface protects business logic. Merging them into one threshold either hurts legitimate users or fails against targeted attacks.
1.6 Rejection Semantics: 503 → 429
limit_reqdefaults to 503 on rejection — semantically wrong: 503 means "service temporarily unavailable" (server fault), 429 Too Many Requests means "you requested too fast" (client fault). One line limit_req_status 429 fixes it, paired with limit_conn_status 429. Why care? Monitoring systems alert on status codes: 5xx spikes trigger "service failure" alerts waking on-call engineers; 429 spikes trigger "traffic anomaly" alerts routed to security/capacity processes. Correct semantics prevent misdirected alerts. Also, rejections aren't free — each 429 consumes a handshake, log write, and CPU; an extremely low rate can turn "rejection" itself into an attack surface (attacker forces you to flood 429s and fill logs), which is why "429 rate spike is itself an event" in monitoring (Section 4.5).
Core Process and Complete Production Configurations
2.1 Complete Config 1: Three-Dimensional Rate Limiting (Global + Site + High-Risk Interface)
This configuration addresses all three incident root causes: whitelist via geo + map, key trust chain (with CDN requires real_ip), three independent zones, 429 semantics, dual request/connection gates. Recommended as a team template:
# ---- http block: define keys and three zones --
geo $whitelist {
default 0;
10.0.0.0/8 1; # office network pass
192.168.0.0/16 1; # internal monitoring probes
}
# whitelist maps to empty key; empty key is not counted nor limited
map $whitelist $limit_key {
1 "";
0 $binary_remote_addr;
}
# With CDN/LB, strip real IP first
# set_real_ip_from 10.0.0.0/8; # trusted proxy CIDR
# real_ip_header X-Forwarded-For;
# Global layer: coarse abuse blocking, loose threshold
limit_req_zone $limit_key zone=global:20m rate=50r/s;
# Site layer: protect specific business domains
limit_req_zone $limit_key zone=order:10m rate=10r/s;
# Interface layer: high-risk endpoints stricter
limit_req_zone $limit_key zone=login:10m rate=3r/m;
# Concurrency gate: paired with rate gate
limit_conn_zone $limit_key zone=conn:10m;
# Rejection semantics: 503 -> 429
limit_req_status 429;
limit_conn_status 429;# ---- server block: three-layer mounting ----
server {
listen 443 ssl;
server_name api.a.com;
# High-risk interface: finest zone first, strictest threshold
location = /api/login {
limit_req zone=login burst=3 nodelay;
limit_req zone=global burst=100 nodelay;
limit_conn conn 10;
proxy_pass http://auth_pool;
}
# Site layer: order domain medium threshold
location /api/order/ {
limit_req zone=order burst=20 nodelay;
limit_req zone=global burst=100 nodelay;
limit_conn conn 20;
proxy_pass http://order_pool;
}
# Fallback: global layer site-wide
location / {
limit_req zone=global burst=100 nodelay;
limit_conn conn 30;
proxy_pass http://web_pool;
}
}Three details determine this config's quality:
Burst magnitude gradient — global burst=100 tolerates page-load concurrency spikes, order burst=20 tolerates double-clicks on checkout, login burst=3 tolerates only hand-tremor retries; the gradient mirrors protection strength.
nodelay on all — gateway rate limiting aims to block abuse, not add latency; scenarios needing queue-based peak shaving belong in message queues.
Whitelist empty-key trick — map maps whitelist to empty string; Nginx does not count/limit empty keys. Cleaner and avoids if inheritance pitfalls.
2.2 Rate Limiting in Action: One Request's Two Fates
Observe how rate=10r/s burst=5 handles a user sending three requests instantly (no prior traffic in last 600ms):
T1: Request 1 arrives, bucket level 0 → passed, level recorded as 1000 (one request's quota in ms).
T1+80ms: Request 2 arrives, level = 1000 - 80×10 + 1000 = 1920, within burst=5×1000 → passed.
T1+120ms: Request 3 arrives, level = 1920 - 40×10 + 1000 = 2880, still below 5000 → passed.
All three pass instantly — that's nodelay daily behavior. Without nodelay, request 3 would be held, waiting 280ms for the bucket to drain, user perceives 280ms extra latency . Without burst, request 2 would be 429-rejected. Same user, same traffic, three configs, three fates. When troubleshooting "rate limiting false positives," always reconstruct this timeline first: check logs for $limit_req_status values PASSED, DELAYED, REJECTED — more intuitive than staring at config.
2.3 Sizing burst: A Ruler Calibrated by Load Testing
burstis not a guess; determine in three steps:
Measure normal peak — from access logs, compute per-IP max requests in 1 second (normal page browsing concurrent loads including static resource API calls). P99 typically 10–30.
Leave attack headroom — set burst to 2–3× normal peak, ensuring 100% of legitimate bursts pass while sustained script over-speed gets caught by the bucket.
Canary observe & calibrate — after rollout, watch DELAYED/REJECTED ratios in logs; both should be near zero for normal traffic. If REJECTED appears, sample rejected IPs to distinguish humans from bots, then fine-tune. Burst is the quantification of "normal user behavior profile," not ops intuition.
2.4 Complete Config 2: Graceful 429 Degradation
Rate-limited requests shouldn't get a bare 503 white page. Standard practice: funnel 429 to a named location returning JSON error body + Retry-After header, enabling client-side backoff retry:
# server block: unified 429 exit
error_page 429 = @toomany;
location @toomany {
default_type application/json;
add_header Retry-After 3 always;
add_header Cache-Control "no-store";
return 429 '{"code":429,
"msg":"requests too fast"}';
}
# Captcha endpoint: dry-run observe before tightening
location = /api/captcha {
limit_req zone=login burst=3 nodelay;
# limit_req_dry_run on; # enable during observation
# limit_req_log_level notice;
proxy_pass http://auth_pool;
}Client discipline is equally critical: on 429, read Retry-After and exponential backoff with jitter (100ms, 200ms, 400ms, max 3 retries + random jitter). Fixed-interval immediate retry turns rate limiting into a retry storm — half the "checkout spinning" in Incident 1 came from frontend mindless auto-retry on 503. Server-side similarly: when upstream rate-limits you, propagate 429 instead of retrying to timeout.
2.5 Validation Three-Steps and Health-Check Scripts
Rate limit config must be validated before going live — three steps:
Dry-run observe — limit_req_dry_run on only logs, doesn't actually reject. With limit_req_log_level notice in staging, see exactly "who would be rejected if live."
Load-test reproduce — ab / wrk targeted beyond limit, confirm 429 appears as expected, whitelist IPs unaffected.
Log statistics — add $limit_req_status to log format; daily patrol REJECTED ratio. Health-check script core lines:
# 429 ratio & top source IPs
awk '{c[$9]++} END{for(s in c) print s, c[s]}' access.log | sort -k2 -rn
awk '$9==429 {print $1}' access.log | sort | uniq -c | sort -rn | head -10
# $limit_req_status distribution (when field present in log)
awk '{print $NF}' access.log | sort | uniq -c | sort -rn
# Load-test verify burst effectiveness
ab -n 2000 -c 50 https://api.a.com/order/2.6 Observation Period & Release Discipline: Two-Phase Rollout
Rate limiting is one of the few defenses where a misconfiguration directly rejects legitimate users. Rollout must be two-phase:
Observation period — new policy runs in dry_run for 3–7 days, only logging. Answer three questions from logs: How many rejected IPs are legitimate users? What percentage of total requests would be rejected? Zone usage at peak? All three healthy (zero false positives, rejections concentrated on obvious abuse, zone usage <60%) → proceed to
Tightening period — remove dry_run, go live, observe 24h confirm 429 rate returns to baseline. Accompanying release discipline: Rate limit parameter changes managed at same rigor as code deployments — review, change ticket, rollback command. Because "accidentally changing rate=30r/m to 30r/s " causes site-wide 429 storm, while rollback is just one reload.
Core Source Code: Four Key Logic Sections
3.1 Leaky Bucket Water Level: excess Calculation
limit_reqcore state is a single number: excess (bucket water level), unit "ms quota" — 1000 = one request's quota. On each request, first drain by elapsed time, then add this request's quota:
ms = (now - node.last) * rate_scale;
excess = node.excess - ms + 1000;
if (excess < 0) excess = 0; // drain empty
if (excess > burst * 1000) {
node.last = now; // reject also accounts
return REJECTED; // over burst limit
}
if (nodelay) {
node.excess = excess; // consume quota
return PASSED; // immediate process
}
if (excess > 1000) {
delay = (excess - 1000) / rate;
return DELAYED; // queue for release
}
node.excess = excess; node.last = now;
return PASSED;Three easily overlooked details:
Reject also updates last — rejected requests advance the timestamp, so a continuously attacking IP stays at max water level; after stopping, must wait for bucket to drain before recovery — this is the leaky bucket's "penalty period."
nodelay only changes release timing, not accounting — quota still consumed, burst exhaustion still rejects; it is not a "bypass."
DELAYED wait time is precisely calculable — (excess-1000)/rate equals milliseconds to wait; this is the source of the 280ms in Section 2.2 timeline.
3.2 Shared Memory and State Reclamation
Zone is a slab shared memory across all workers; key states organized as hash + red-black tree, O(1) lookup. Reclamation mechanism:
States with no requests for >60 seconds are lazily reclaimed (memory reused for new keys). Two implications: (1) Public entry IP count far exceeding zone capacity won't immediately "fill" — many IPs are one-off, reclaimed in 60s. But under CC attack, hundreds of thousands of fake IPs stay active simultaneously, zone truly fills, new keys cannot allocate and are treated as over-limit; attacker effectively "bursts your ledger" — hence public zones must be large and monitored. (2) Key state only grows is normal; don't expect zone to "auto-shrink."
3.3 limit_conn Reference Counting
// Request entry: lookup/create counter node
node = lookup(zone, $limit_key);
if (!node) node = alloc(zone, 1);
else node.conn++; // ref +1
if (node.conn > conn_limit) {
if (created) free(node);
return 429; // over concurrency -> reject
}
// Request end (log phase) releases:
node.conn--; // forget to decrement = leak
if (node.conn == 0) mark_free(node);limit_conndecrements in the log phase , explaining two behaviors: (1) Long-lived connections (WebSocket, SSE) permanently occupy a slot — concurrency limit acts as a "resident gate" for them, requiring separate business-specific thresholds. (2) If error handling path skips log phase (extreme crash), counter leaks, making that IP permanently "full" — restart required to recover. Rare, but when debugging "specific IP all 429, only restart helps," recall this.
3.4 Multiple limit_req Stacking Rules
Multiple limit_req directives in one location (as in Config 2.1) are legal and recommended:
Each uses its own zone independently; any single rejection rejects the request. Thus interface layer (strict) + global layer (loose) stacking means normal requests constrained by the strictest, attack requests caught by the first that rejects.
Inheritance trap: if current level defines any limit_req , it stops inheriting all parent-level limit_req directives. If /api/order/ only writes order zone, the server-level global zone becomes ineffective for that path — attackers bypassing /api/order/ rate hit other paths with no global defense. When troubleshooting "rate limit configured but not working," check this inheritance rule first.
Advanced Topics: Five Accounts to Settle Upfront
4.1 Cluster Rate Limiting: Three Solutions Trade-offs
Incident 2 proved per-node independence limits. Three solutions by increasing cost:
Derive per-node from total — global target 100r/s, four nodes → each 25r/s with fair scheduling. Zero code change, but node scaling requires config change; traffic imbalance still causes skew.
Redis + Lua atomic counter — all nodes hit same Redis key for atomic window counting. True global, cost: extra Redis hop per request (1–2ms intra-DC; high concurrency needs local pre-deduction). Redis failure needs fallback policy (fail-open recommended).
OpenResty resty.limit.req — embedded Lua implements centralized leaky bucket, semantics identical to limit_req with multi-level stacking support. Standard answer for "need both global and leaky bucket."
Selection rule: if rate limiting is just abuse prevention, option 1 suffices; if it's a business capacity gate (e.g., seckill admission), must use 2 or 3.
4.2 Identity-Based Rate Limiting: From IP to User
IP dimension has two inherent flaws: large-exit NAT (campus, corporate) aggregates hundreds of users into one IP — tight limit causes mass false positives; distributed proxy pools give attackers unlimited IPs — loose limit renders it useless. Solution: switch key to "identity" — authenticated endpoints use uid/token (map $http_authorization or custom header), combined with IP layer for dual defense — IP layer blocks unauthenticated scripts, identity layer blocks authenticated abuse; unauthenticated endpoints use "IP + device fingerprint header" combo. Note identity key memory overhead: uid states larger than IP, zone must scale accordingly; logged-out token records also follow 60s lazy reclamation — account for this in capacity planning.
4.3 Layered Division: Gateway Rate Limiting Is Not Everything
Complete rate limiting system has four layers each with distinct duty:
CDN/WAF layer — intercepts flood before reaching origin (CC protection, geo blocking); first and most efficient filter.
Gateway layer (this article) — IP/identity rate & concurrency control; protects total backend service capacity.
Business layer — rate limits by business semantics (one order per user per second, inventory deduction rate); gateway cannot see business semantics, this must live in application.
Data layer — connection pool caps, slow query circuit breakers; final resource safety net.
Common fallacy: "gateway rate limiting configured → system stable." In seckill, true admission control is business-layer inventory + queue; gateway only keeps malicious traffic outside — don't make gateway carry business capacity burden.
4.4 Six High-Frequency Pitfalls Checklist
Treating rate as concurrency — rate=10r/s misunderstood as "10 simultaneous requests"; actually it's rate; use limit_conn for concurrency.
Inheritance override blind spot — location defines a limit_req, server-level ones stop inheriting; multi-layer defense must explicitly stack at each layer (Section 3.4).
Whitelist via if — if wrapping limit_req fails or behaves weirdly; correct pattern: map to empty key.
Burst without nodelay — after rollout users complain "API slowed 300ms"; actually queueing delay, not performance regression.
NAT large-exit false positives — entire campus 429; fix: identity-based limiting or whitelist that CIDR then use alternate key dimension.
429 can also flood logs — under heavy attack access_log becomes all 429, disk I/O saturated; consider limit_req_log_level and log sampling for rate-limiting scenarios.
4.5 Monitoring & Alerting: 429 Rate Is a Sensitive Metric
Rate limiting must enter monitoring; three handles:
429 rate baseline & spike — normal business 429 rate <0.1%; spike >1% indicates attack, misconfiguration, or upstream capacity collapse triggering retries — all warrant immediate log review.
$limit_req_status distribution — PASSED/DELAYED/REJECTED/ABORTED four-state ratios; persistent non-zero REJECTED → sample audit rejected IPs.
Zone usage — zone saturation manifests as all new IPs rejected; public entries during promotions need extra watch.
Alert suggestion: 429 rate >0.5% warning, >5% critical. Daily patrol runs Section 2.5 health-check script in daily report; rate limiting shifts from "configure and forget" to "continuously controlled."
4.6 Big Promotion Rate Limiting Plan: Six-Step Walkthrough
One week before major promotion, rate limiting plan executes fixed six-step walkthrough:
Recalculate thresholds — promotion normal peak 3–10× daily; global burst recalculated per new P99, err on loose side.
Calibrate whitelist — load-test engine egress IPs, monitoring probe nodes, partner callback addresses all whitelisted; missing one = "self-DDoS" on promotion day.
Pre-warm zones — promotion start IP count surges; zone capacity verified against estimated active IPs × 64B.
Drill degradation chain — trigger real 429, confirm frontend receives JSON + Retry-After and follows backoff strategy, not white page.
On-call handbook — codify "false positive emergency triad": sample rejected IP profiles → whitelist or adjust param → reload; all within 10 minutes.
Post-mortem — compare planned thresholds vs actual traffic; each false positive in REJECTED traced to root cause, feeds next cycle's profiling. Essence: shift Section 2.3 "user behavior profile" from daily to promotion scenario; skipping any step risks the costliest accident at peak traffic.
Learning Path: From Configuring to Defending
Internalize semantics — read official ngx_http_limit_req_module and limit_conn docs line by line; locally reproduce rate/burst/nodelay three-combination behavior differences (one ab command reveals all).
Build one three-dimensional defense — apply Section 2.1 template to your project, run dry-run, screenshot DELAYED/REJECTED log distribution for archive.
Understand boundaries — read limit_req source excess calculation and zone reclamation ( ngx_http_limit_req_module.c <1000 lines); grasp "reject also accounts" and "60s reclamation"; compare with Sentinel/Resilience4j limiters to feel gateway vs business layer division.
Advance to cluster rate limiting — implement global rate limiting with OpenResty resty.limit.req or Redis+Lua; load-test compare "per-node independent" vs "global shared"; complete mindset shift from single-node to cluster.
High-Frequency Interview Questions (with Answers)
The article includes 20 Q&A pairs covering algorithm, parameters, cluster, key trust, inheritance, monitoring, etc. Key examples:
Q1: Algorithm & core params? Leaky bucket; rate=drain speed, burst=bucket depth, nodelay=process queued immediately.
Q2: Is rate=10r/s "max 10 per second"? No — constant 1 per 100ms; window counter allows burst, leaky bucket doesn't.
Q3: burst+nodelay three combos behavior? Only rate: instant reject; rate+burst: queue delay; rate+burst+nodelay: immediate process, quota accounted, reject when exhausted. API gateways pick third.
Q4: limit_req vs limit_conn difference & pairing? limit_req=rate (anti-flood), limit_conn=concurrency (anti-occupation); slow attacks need limit_conn, flash floods need limit_req; both mandatory.
Q5: Why binary_remote_addr? Same semantics as remote_addr but 4-byte binary vs up to 15-byte string; smaller zone footprint, fixed-length keys. 10m zone ≈160k IP states.
Q6: Problem with X-Forwarded-For as key? Client-forgeable header; attacker rotates header = new user; limit fails. Correct: direct use binary_remote_addr; with CDN/LB use real_ip module to peel trusted proxies.
Q7: N nodes each 10r/s → global limit? ~N×10r/s; state per-node independent; true global needs derived per-node rates or Redis/OpenResty centralized limiting.
Q8: Reject 503 or 429? Why? 429. 503=server fault (triggers service alerts); 429=client fault (traffic anomaly flow). limit_req_status 429 fixes; add Retry-Ahead for client backoff.
Q9: Whitelist office/monitoring? geo defines whitelist flag, map maps to empty key — Nginx doesn't count/limit empty keys. Avoid if wrapping.
Q10: Multiple limit_req in same location execution? Each independent zone; any reject → reject. Inheritance: defining any limit_req at current level blocks all parent limit_req inheritance; must explicitly stack each layer.
Q11: When does limit_conn release count? Log phase. Long connections (WebSocket/SSE) permanently occupy slots → separate thresholds; extreme error paths skipping log phase leak counters → IP forever 429 until restart.
Q12: What is excess? "Reject also accounts" meaning? excess=water level (1000=1 request ms quota); reject updates last timestamp and occupies level; continuous attacker stays maxed; after stop must wait bucket drain — leaky bucket penalty period.
Q13: How to size burst? Three steps: log per-IP 1-sec max requests P99 = normal peak; burst = 2-3× peak for attack headroom; canary observe DELAYED/REJECTED ratios calibrate. Essence: quantify normal user behavior profile, not intuition.
Q14: Campus/corporate NAT mass false positives? NAT aggregates many users per IP. Fix: authenticated endpoints use uid/token key (identity limiting), unauthenticated use IP+device fingerprint; or whitelist CIDR then alternate dimension.
Q15: Client receives 429 — what to do? Read Retry-After, exponential backoff with jitter (100/200/400ms, max 3 retries), degrade after limit. Forbid fixed-interval immediate retry — amplifies rate limit into retry storm. Server upstream-limited should propagate 429 not hard-retry.
Q16: limit_req configured but not effective — common causes? 1) location defines limit_req blocking parent inheritance; 2) key source wrong (XFF forge / all proxy IP collapsed); 3) whitelist misroutes traffic to empty key; 4) per-node config, single node rate not hit but aggregate exceeds. Check each.
Q17: rate=30r/m behavior? One request every 2 seconds (60/30) constant rate; not "first 30 free per minute". Low rates must use r/m for clarity and avoid r/s magnitude disasters.
Q18: Metrics to monitor? Alert thresholds? 429 rate (normal <0.1%, >0.5% warn, >5% critical); $limit_req_status four-state distribution (persistent REJECTED → sample audit); zone usage (promotion prevent ledger saturation). Health-check script in daily report.
Q19: What is dry-run? When used? limit_req_dry_run on logs only, no reject; with limit_req_log_level notice shows "who would be rejected if live." Used for observation period before new policy goes live to avoid false-positive incidents.
Q20: Gateway vs business rate limiting division? Four layers: CDN/WAF blocks flood; gateway IP/identity rate+concurrency (protects service total); business layer business-semantic limits (one order/user/sec, admission control); data layer connection pool/slow query circuit breaker. Gateway sees no business semantics — seckill admission always business layer; don't burden gateway with business capacity.
One-Page Cheatsheet
Trio: rate=drain speed, burst=bucket depth, nodelay=no queue; API gateway standard = burst+nodelay. Semantics: rate is constant rate not window counter; 10r/s = 1 per 100ms. Dual gates: limit_req anti-flood, limit_conn anti-occupation; must pair. Key: trust only binary_remote_addr and real_ip peeled true IP; XFF forgeable; whitelist via map empty key. Defense: global coarse + site medium + interface fine three-layer stacking; current level has limit_req → no parent inheritance. Status code: limit_req_status 429 + Retry-After + client exponential backoff. Cluster: per-node independent; global needs Redis/OpenResty. Monitoring: 429 rate + REJECTED distribution + zone usage; >0.5% alert.
Summary & Next Episode Preview
All rate limiting difficulty lies in four words: "semantic precision": rate is rate not concurrency, burst is capacity not switch, nodelay trades latency not quota, reject also accounts, nodes each independent. Two complete configs providing three-dimensional defense and 429 degradation, combined with dry-run observation period and health-check scripts, cover most scenarios from abuse prevention to business capacity protection; remaining cluster rate limiting handed to Redis and OpenResty in Episode 09. Next episode enters encryption chain: HTTPS and kernel tuning — certificate chains & TLS handshake cost, session reuse, OCSP stapling, and the checklist pushing Nginx and Linux kernel parameters to the limit.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
