How Our Reactive API Gateway Powers Microservices: Architecture & Features

With the rise of microservices, API gateways have become an indispensable component, serving as the sole entry point for external traffic while providing many common functionalities.

Overall Architecture

Our API gateway sits at the top, handling all north‑south traffic and routing it downstream to microservice applications or BFF aggregation services. Before BFF standardization, it is treated as a regular microservice.

The gateway currently implements request dispatch, conditional routing, API management, rate limiting, circuit breaking, security policies, monitoring alerts, and call‑chain tracing.

Built on RxNetty, the gateway follows an asynchronous reactive model, achieving high single‑node concurrency. Most features are assembled from existing platforms: request dispatch and conditional routing from the microservice framework, rate limiting and circuit breaking from the stability platform, monitoring from the monitoring platform, and security policies from the big‑data analysis platform. Service registration and configuration are handled by separate registry and config centers.

Request Dispatch

Dispatching requests is the most basic gateway function. Unlike typical Nginx‑based gateways that update upstreams dynamically, our gateway treats itself as a subscriber‑only microservice, using the service discovery capabilities of the existing microservice framework to match URLs to namespaces and route to the appropriate service.

After URL normalization, each application registers a prefix in the registry. The gateway builds a namespace for each prefix, enabling fast matching and routing using the framework’s routing and load‑balancing features.

Service discovery objects are managed via a control platform, allowing dynamic addition or removal of services through hot updates of the in‑memory cache.

Conditional Routing & Gray Release

Conditional routing filters requests based on specific content or traffic proportion, enabling gray releases, blue‑green deployments, and A/B testing.

Our implementation reuses the existing microservice framework: each application can define groups of instances, and the gateway creates Invoker objects with separate spaces for each group. Requests are matched against these spaces to decide routing.

We currently support matching by request content (e.g., headers, attributes) or traffic ratio, primarily using a "version" label for instance grouping, with future plans to support Kubernetes pod labels.

Combined with the DevOps platform, this allows easy gray releases, such as routing requests from user ID 100 to a test version.

API Management

The "API" prefix highlights the gateway’s API management capability, distinguishing internal microservice calls from external client calls.

API definitions are stored in a database; when users publish APIs via the control platform, the data is saved and pushed to the gateway through the config center. A filter removes deleted or unpublished APIs during namespace matching, and hot updates refresh the filter.

Improved user experience includes:

Direct synchronization of newly added APIs from the microservice control platform to the gateway without manual steps, supporting multiple file formats.

Cross‑environment API promotion without duplication.

Filtering and displaying APIs by status.

Integration with the DevOps platform to remind users to publish API changes during application releases.

Rate Limiting & Circuit Breaking

As the sole entry for north‑south traffic, the gateway must manage high concurrency and complex traffic patterns.

Both rate limiting and circuit breaking are implemented via a stability platform built on a customized Sentinel, with configurations distributed through the config center and applied via hot updates.

Rate limiting controls QPS, isolation controls concurrent requests; circuit breaking monitors error rates and triggers fallback responses. Metrics are exposed via Prometheus for monitoring.

Security Policies

To mitigate malicious traffic such as web crawlers, basic security policies are essential.

Security rules are configured manually via the gateway control platform and pushed through the config center to the gateway’s securityControl component, which evaluates incoming requests and exposes metrics for blocked traffic.

Additionally, real‑time log collection feeds a big‑data analysis platform that can automatically generate blocking rules for suspicious IPs or users and trigger alerts.

Supported rule criteria include client IP, user ID, and other HTTP headers/attributes; actions include fast‑fail and CAPTCHA challenges.

Monitoring, Alerts & Tracing

The gateway integrates monitoring, alerting, call‑chain tracing, and log querying similar to other microservices.

Metrics are exposed via an endpoint for the monitoring system; tracing SDK records call‑chain data; logs are collected and sent to the monitoring center. Alerts can be configured for metric thresholds, error logs, or host‑level issues.

When backend services fail (e.g., timeouts, connection pool exhaustion), the gateway detects the error and triggers alerts for both the gateway owner and the backend service owner via a dual‑alert mechanism.

Summary

The API gateway offers many features not fully covered here, such as WebSocket support and multi‑cloud traffic scheduling. Future improvements include validating high‑concurrency performance, introducing a rule engine for security, stability, and routing policies, and expanding security criteria to include IP ranges and complex logical expressions.

Source: https://fredal.xin/build-api-gateway