How Our Reactive API Gateway Powers Microservices with RxNetty

Overall Architecture

With the rise of microservices, the API gateway has become an indispensable component, serving as the sole entry point for external traffic and extracting many common functions for applications.

Our API gateway architecture is shown below, positioned to handle all north‑south traffic and dispatch requests to microservice or BFF aggregation applications. Before BFF standardization, it is treated as a regular microservice.

The gateway currently provides request distribution, conditional routing, API management, rate limiting, circuit breaking, security policies, monitoring, alerting, and tracing.

Implemented with RxNetty, the gateway follows an asynchronous reactive model, achieving high single‑node concurrency. Most features are built by reusing existing platform components: request distribution and conditional routing rely on the microservice framework, rate limiting and circuit breaking on a stability platform, monitoring on a monitoring platform, and security policies on a big‑data analysis platform. Service registry and configuration center handle service registration and third‑party configuration distribution.

Request Distribution

Request routing is the gateway’s most basic function. Unlike traditional Nginx‑based gateways that update upstream proxies dynamically, our gateway treats itself as a subscriber‑only microservice. It receives only URL information, allowing us to reuse existing service‑discovery mechanisms.

After URL normalization, each application registers a unique prefix in the registry. The gateway builds a namespace object per prefix, enabling fast matching of incoming requests to the appropriate microservice without additional code.

Service discovery objects are managed centrally; users can control the online/offline status of microservices via a control platform, which pushes updates through the configuration center for hot‑reloading.

Conditional Routing & Gray Release

Conditional routing filters requests with specific content or traffic ratios and routes them to particular instance groups, forming the basis for gray releases, blue‑green deployments, and A/B testing.

Our implementation reuses the microservice framework: each application defines groups (e.g., by version). During service discovery, an Invoker creates a Space for each group; the gateway matches request rules against these Spaces to decide routing.

We support matching by request headers, attributes, or traffic proportion, currently focusing on version‑based grouping with future plans to incorporate Kubernetes pod labels.

API Management

The gateway includes API management to distinguish internal microservice calls from external client calls. API definitions are stored in a database; publishing actions on the control platform are persisted and propagated via the configuration center.

A filter layer removes unpublished or offline APIs during namespace matching, and hot‑updates keep the filter in sync.

Synchronize newly added APIs from the microservice control platform directly to the gateway without manual steps.

Support importing APIs in multiple file formats.

Filter and display APIs by status.

Integrate with the DevOps platform to trigger API publishing during application release flows.

Rate Limiting & Circuit Breaking

Given the gateway’s role as the sole entry point, high concurrency and complex traffic patterns necessitate robust traffic control.

Both rate limiting/isolation and circuit breaking are implemented via a stability platform built on a customized Sentinel, with configurations distributed through the configuration center for hot‑updates.

Rate limiting controls QPS, while isolation limits concurrent requests. Circuit breaking monitors error rates on outbound calls, triggering fallback responses when thresholds are exceeded. Metrics are exposed via Prometheus‑compatible SDKs for real‑time monitoring.

Security Policies

To mitigate malicious traffic such as web crawlers, the gateway provides configurable security rules.

Users define rules in the gateway control platform; these are pushed to the gateway’s securityControl component for hot‑reloading. Violations are logged and exposed as metrics.

Additionally, logs are streamed to a big‑data analysis platform that can automatically generate block rules for suspicious IPs or users, supporting actions like fast‑fail or CAPTCHA challenges.

Monitoring, Alerting & Tracing

The gateway integrates metrics, tracing, and logging similar to other microservices. Metrics are exposed via an endpoint for the monitoring system; tracing SDK records call chains; logs are collected centrally.

Alerts can be configured for metric thresholds, error logs, or host‑level issues. When backend services fail (e.g., timeouts), the gateway generates alerts for both the gateway owner and the downstream service owner.

Summary & Future Work

The gateway also supports WebSocket protocols and serves as a multi‑cloud traffic scheduler, though these aspects are not detailed here.

High‑concurrency capabilities have not been fully validated in production.

Consider integrating a rule engine to manage security, stability, and routing rules.

Expand security policies to include IP ranges and complex logical expressions.