How Our Reactive API Gateway Handles Routing, Rate Limiting, and Security in Microservices

Overall Architecture

The API gateway sits at the edge of the microservice ecosystem, acting as the sole entry point for north‑south traffic and forwarding requests to downstream microservices or BFF aggregation services. It provides request dispatch, conditional routing, API management, rate limiting, circuit breaking, security policies, monitoring, alerting, and tracing.

The gateway is implemented with RxNetty, providing an asynchronous reactive processing pipeline that achieves high single‑node concurrency. Most features are assembled from existing platform components: request dispatch and conditional routing reuse the microservice framework, rate limiting and circuit breaking rely on a stability platform, monitoring uses a monitoring platform, and security policies draw on a big‑data analysis platform. Service registration and configuration are handled by separate registry and config centers.

Request Dispatch

Request dispatch is the most basic gateway function. Unlike traditional Nginx‑based gateways that update upstream lists dynamically, our gateway treats itself as a non‑registering microservice that only receives URL information. After URL normalization, each application registers a unique prefix in the registry. The gateway builds a namespace object for each prefix and uses existing microservice SDKs for routing and load balancing.

Service discovery is managed via a control panel where users can enable or disable microservice applications at the gateway layer. Changes are pushed through the configuration center, triggering a hot update that refreshes the in‑memory cache.

Conditional Routing & Gray Release

Conditional routing filters requests based on specific content (e.g., headers, attributes) or traffic proportion, enabling gray releases, blue‑green deployments, and A/B testing. Each application can define groups of instances; during service discovery the gateway creates an Invoker for each application, which in turn creates Space objects for each group. At request time, the gateway matches the request against these spaces to decide the target group.

Currently supported matching dimensions are request content (headers, attributes) and traffic ratio, with grouping primarily based on a "version" label. Future work includes supporting Kubernetes pod labels.

API Management

The gateway also functions as an API management platform, distinguishing internal microservice calls from external client calls. API definitions are stored in a database; publishing actions in the control panel write to the DB and trigger a pub/sub notification via the configuration center. A filter layer removes unpublished or deleted APIs before namespace matching, allowing hot updates of the filter.

Synchronize newly added APIs from the microservice control platform to the gateway without manual steps.

Support multi‑environment API promotion to avoid duplicate entries.

Filter and display APIs by status.

Integrate with the DevOps platform to trigger API‑related release reminders.

Rate Limiting & Circuit Breaking

High concurrency and complex traffic patterns at the gateway necessitate robust flow control. Both rate limiting/isolation and circuit breaking are implemented via a stability platform built on a customized Sentinel. Rate limiting controls QPS, isolation limits concurrent requests, and circuit breaking triggers on error‑rate thresholds, optionally returning fallback data.

All rules are configured in the stability platform, distributed via the configuration center, and hot‑updated in the gateway. Sentinel SDK collects metrics and exposes them to Prometheus for monitoring.

Security Policies

To mitigate malicious traffic such as web crawlers, the gateway provides configurable security policies. Administrators can manually define rules in the control panel, which are pushed to the gateway's securityControl component for hot updates. Real‑time log collection feeds a big‑data analysis platform that can automatically generate blocking rules and trigger alerts.

Supported criteria include client IP, user ID, and arbitrary HTTP headers/attributes. Actions can be fast‑fail or redirect to a CAPTCHA page.

Monitoring, Alerting & Tracing

The gateway integrates with the existing monitoring stack to expose metrics, tracing, and logs. Metrics are scraped by the monitoring center, tracing SDK records spans, and logs are collected for centralized analysis. Alerts can be configured for metric thresholds, error logs, or host‑level issues.

When backend services fail (e.g., timeout, connection pool exhaustion), the gateway generates alerts for both the gateway owner and the affected service owner, enabling dual‑side incident response.

Summary & Future Work

The gateway currently supports WebSocket, multi‑cloud traffic scheduling, and a range of core functions, but several areas need improvement:

High‑concurrency performance has not been stress‑tested in production.

Introduce a rule engine to handle diverse policies (security, stability, routing).

Extend security policies to support IP ranges and complex logical expressions (AND/OR/NOT).