How KubeGateway Solves kube‑apiserver Load Imbalance and Boosts Request Governance

Why Build KubeGateway

In a Kubernetes cluster, kube‑apiserver is the entry point for all resource operations, so its high availability determines the cluster's overall reliability. Traditional setups deploy multiple kube‑apiserver instances behind external load balancers, but these often cause uneven traffic distribution and lack application‑layer request governance.

KubeGateway Architecture Design

KubeGateway is a seven‑layer gateway tailored for kube‑apiserver traffic. Its key features include:

Transparent to clients—no changes required on the client side.

Supports proxying multiple clusters, distinguished by domain or virtual IP.

Load balancing at the HTTP request level, solving load imbalance.

Pluggable load‑balancing strategies (Round Robin, Random) with easy extensibility.

Flexible routing based on resource, verb, user, namespace, API group, etc., enabling fine‑grained traffic control.

Configuration managed via standard Kubernetes APIs with hot‑update support.

Built‑in capabilities such as rate limiting, degradation, service discovery, graceful shutdown, and upstream health checks.

The gateway processes requests in five steps: request parsing, route matching, user authentication, traffic governance, and reverse proxy.

Request Parsing

KubeGateway distinguishes between resource requests (e.g., CRUD on Pods) and non‑resource requests (e.g., /healthz, /metrics). For resource requests it extracts multi‑dimensional routing fields from URLs and headers.

Route Matching

Extracted routing fields enable powerful rules, such as matching all list‑pod requests by verb + resource or isolating controller‑manager traffic by user/group, allowing precise load‑balancing, rate limiting, and circuit breaking.

User Authentication

KubeGateway forwards user identity to upstream kube‑apiserver using impersonation. It supports x509 client‑certificate authentication and Bearer‑Token authentication via TokenReview.

Impersonate-User: Client username<br/>Impersonate-Group: Client group

Upstream kube‑apiserver validates the impersonation and performs authorization.

Traffic Governance

Features include load balancing (Round Robin, Random, extensible), health monitoring of /healthz, rate limiting (token bucket and max‑in‑flight requests), and degradation to protect the cluster during failures.

Reverse Proxy & Impersonation

During proxying, KubeGateway injects Impersonate‑User and Impersonate‑Group headers so that upstream kube‑apiserver executes requests on behalf of the original client.

HTTP/2 Multiplexing

KubeGateway uses HTTP/2 by default, allowing up to 250 streams per connection, dramatically reducing the number of TCP connections to upstream kube‑apiserver.

Forward & Exec Request Handling

For requests like forward or exec that require HTTP 1.1 or other protocols, KubeGateway disables HTTP/2 and supports hijacking to maintain transparency.

KubeGateway Impact

Performance tests show only ~1 ms added latency, handling over 200 k QPS across ByteDance’s clusters. It resolves kube‑apiserver traffic imbalance, adds request governance (grouping, routing, rate limiting, degradation), and improves cluster stability.

Future Evolution

Extend seven‑layer capabilities (black‑white lists, caching).

Enhance observability for faster issue diagnosis.

Explore federation by aggregating multiple clusters through the gateway.