Why Add an API Gateway for Internal Services When Direct Connection Is Possible?

Service Governance Challenges

With ~50 micro‑services, embedding rate‑limiting, circuit‑breaking, logging and gray‑release logic in each service creates a maintenance nightmare. Two concrete problems are highlighted:

Version mismatch : Service A uses a v1.0 rate‑limiting component while Service B has upgraded to v2.0 with different configuration parameters.

Language barrier : Core flow in Java, high‑performance modules in Go, algorithm modules in Python – building identical governance SDKs for each language is impractical.

The essence of an API gateway is to extract these cross‑cutting concerns from business code so that inbound traffic, regardless of backend language, passes a unified layer for rate‑limiting, TraceID tagging and monitoring.

Gray Release

Production avoids full‑scale rollout by using canary or gray testing, e.g., routing 1 % of traffic to a new version B and 99 % to the old version. Direct service‑to‑service calls require the caller to implement complex load‑balancing and routing logic, tightly coupling business and deployment concerns.

The gateway makes this elegant: a configuration‑center‑driven routing rule can shift traffic based on Header version, UserID or custom tags, without any change to business code.

Protocol Adaptation

Different teams expose services with heterogeneous protocols: legacy services may use SOAP or RESTful APIs, while newer services adopt gRPC or Dubbo. Direct connections would force each client to handle protocol conversion and manage additional dependencies.

The gateway acts as a translator, exposing a unified RESTful interface externally while internally converting to the appropriate protocol (e.g., HTTP → gRPC). Callers therefore do not need to know the implementation language or protocol of the callee.

Security and Decoupling

Assuming the internal network is perfectly secure is a false premise. Without a gateway, every microservice endpoint is exposed internally; a leaked port or weak permission allows any internal node to scan and attack core services.

The gateway hides actual IPs and ports, exposing only a virtual URL, and converges service interfaces. Admin‑only endpoints (e.g., cache clear, data export) can be restricted at the gateway layer without code changes.

High‑Concurrency Traffic Buffering

In a direct model, a sudden surge from Service A to Service B can overwhelm B, causing crashes before local rate‑limiting takes effect. Although B can apply local limits, the TCP connection and resources are already consumed.

At the gateway layer, global rate‑limiting can be enforced before traffic reaches backend services, intercepting excess or abnormal requests. This front‑line protection is critical during large‑scale events such as sales promotions.

Common Concerns

Adding a gateway introduces a few milliseconds of latency and creates a single point of failure. Therefore, high‑availability clusters with multiple replicas are required.

Despite these costs, the trade‑off—unified governance, seamless gray release, protocol decoupling, and security convergence—delivers a high cost‑performance ratio.

Recommendation

For tiny projects with only a few services, direct connection may be simpler. Once the number of services exceeds ten and cross‑team collaboration, frequent iteration, or diverse protocols are needed, an API gateway becomes essential.