API Gateway Architecture and Core Functionalities Overview

Overall Architecture

With the popularity of micro‑services in recent years, API gateways have become an indispensable part of micro‑service architecture. They serve as the sole external entry point and extract many common functionalities.

Our API gateway architecture is shown in the diagram below. The gateway sits at the top, handling all north‑south traffic, and forwards traffic downstream to micro‑service applications or BFF aggregation services. Before BFF standardization we treat it as a regular micro‑service.

Request Distribution

The most basic function of a gateway is request distribution. Unlike Nginx‑based gateways that dynamically update upstreams, our implementation treats the gateway as a micro‑service that only subscribes to service discovery. URLs are normalized and prefixed per application; the prefix information is registered in the service registry. During request matching, the gateway builds a namespace object for each prefix and selects the corresponding micro‑service.

Service discovery objects are managed via a control platform, allowing dynamic on‑line/off‑line of applications. Updates are pushed through the configuration center and hot‑reloaded in memory.

Conditional Routing

Conditional routing filters requests with specific content or traffic proportion and routes them to particular instance groups, enabling gray releases, blue‑green deployments, A/B testing, etc.

Our implementation reuses the existing micro‑service framework. Each application can define groups; during service discovery the gateway creates an Invoker with multiple Space objects representing these groups. At request time, the Space objects are matched against routing rules to decide the target group.

We currently support matching by request content (e.g., HTTP headers, attributes) or traffic ratio. Grouping is primarily based on a "version" label, with future plans to support Kubernetes pod labels.

API Management

API management is essential to distinguish internal micro‑service calls from external client calls. API definitions are stored in a database; publishing actions from the control platform are pushed via the configuration center. A filter layer removes unpublished or deleted APIs before namespace matching, enabling hot updates.

Key user‑experience features include:

Direct synchronization of newly added APIs from the micro‑service control platform to the gateway control platform, with support for various import file formats.

Cross‑environment API release without duplicate configuration.

State‑based filtering and display.

Integration with DevOps platforms to trigger API release reminders during application deployment.

Rate Limiting / Isolation & Circuit Breaking

Given the high concurrency and traffic complexity of north‑south traffic, we implement rate limiting, isolation, and circuit breaking using a stability platform built on a customized Sentinel and the configuration center.

Rate limiting controls QPS, isolation controls concurrent connections, and circuit breaking limits outbound traffic based on error rates, providing fallback responses. Rules are configured in the stability platform, pushed via the configuration center, and applied in real time. Metrics are exposed to the monitoring platform via Prometheus.

Security Policies

To mitigate abnormal traffic such as malicious crawlers, we provide basic security policies configurable through the gateway control platform and pushed via the configuration center.

Rules can be manually set or automatically generated from real‑time log analysis on a big‑data platform. Supported criteria include client IP, user ID, HTTP headers/attributes, with actions such as fast‑fail or captcha challenges.

Monitoring, Alerting & Tracing

The gateway integrates Metrics SDK, tracing SDK, and logging collection to provide full observability. Metrics are exposed via an endpoint for the monitoring center; tracing logs are sent to the tracing system; both are also available to the logging system.

When backend services experience errors (e.g., timeouts, connection pool exhaustion), the gateway generates alerts for both the gateway owner and the backend service owner via a dual‑alert mechanism.

Some Summary

Additional capabilities not covered in detail include WebSocket support and multi‑cloud traffic scheduling.

Future improvement directions:

Validate high‑concurrency performance in real‑world scenarios.

Introduce a rule engine to handle diverse policies (security, stability, routing).

Extend security policies to support IP ranges and complex logical expressions.

Source: https://fredal.xin/build-api-gateway