How Alibaba Evolved Envoy Gateway from 1.0 to 3.0: A Cloud‑Native Journey

Background and Motivation

In 2018 Alibaba began a cloud‑native transformation to unify the middleware stack of Alibaba and Ant Group. Divergent RPC protocols, long inter‑service call chains, high protocol‑conversion cost, and limited service‑mesh capabilities of the existing Tengine gateway created the need for a more extensible, high‑performance gateway solution.

Evolution of Envoy Gateway

Envoy Gateway 1.0 (Incubation – May 2020)

Envoy was selected for its extensibility and rapid community adoption. Version 1.0 focused on east‑west RPC traffic between services. The architecture placed Envoy as a sidecar‑less gateway that routes traffic directly to pod IPs, bypassing the traditional ClusterIP service abstraction. This design reduced round‑trip latency and enabled the platform to handle the Double‑11 shopping festival traffic, reaching tens of thousands of TPS.

Envoy Gateway 2.0 (Growth – Dec 2020)

To support north‑south traffic and hybrid‑cloud RPC scenarios, a two‑layer architecture was introduced. Tengine remained as the traffic‑gateway front‑end, while Envoy acted as the micro‑service gateway handling service‑level routing, traffic governance, and protocol translation. This allowed seamless integration with external systems such as DingTalk, Alibaba Video Cloud, and digital‑human services. The Youku use‑case demonstrated consolidation of multiple second‑layer micro‑service gateways into a single Envoy instance.

Envoy Gateway 3.0 (Maturity – 2021)

Version 3.0 merged the two layers into a single Envoy‑based gateway. The unified design eliminated the need for a separate Tengine front‑end, simplifying deployment and operation. Performance was further boosted by:

Hardware TLS acceleration, increasing HTTPS QPS by roughly 80%.

Kernel‑level socket and packet processing optimizations.

Integration with the MSE cloud‑native gateway for unified management across Alibaba’s middleware portfolio.

Key Technical Features

Direct PodIP routing : Traffic is sent straight to the target pod IP, bypassing the ClusterIP service abstraction and reducing network round‑trip time.

HTTPS hardware acceleration : Off‑loading TLS termination to dedicated hardware improves throughput and reduces CPU load, yielding an ~80% QPS increase.

Wasm plugin marketplace : Envoy’s WebAssembly support enables hot‑loading of custom plugins written in multiple languages, allowing runtime extension of routing, security, or observability logic without redeploying the gateway.

Multi‑Ingress Controller : A self‑developed controller reuses a single Envoy instance across multiple Kubernetes clusters, simplifying multi‑cluster ingress management and reducing resource duplication.

Kubernetes Ingress compatibility : Native support for the standard K8s Ingress API and automatic translation of Nginx‑style annotations ensure seamless migration from existing Nginx ingress controllers.

Community Contributions

During the evolution of Envoy Gateway, the team contributed upstream issues and patches to the Envoy project, including enhancements for the dubbo_proxy filter, improvements to Wasm integration, and security hardening via the cryptomb module. These contributions help the broader community adopt Envoy for high‑performance, cloud‑native gateway scenarios.