Key Terminology and Architecture Overview of the Envoy Proxy

Terminology

Before diving into the main architecture documentation, several definitions are introduced that may be controversial in the industry but are used consistently throughout Envoy's documentation and codebase.

Host : An entity capable of network communication (e.g., an application on a phone or server). In this document, a host is a logical network application; a single physical machine may run multiple hosts as long as they can be addressed independently.

Downstream : A downstream host connects to Envoy, sends requests, and receives responses.

Upstream : An upstream host receives connections and requests from Envoy and returns responses.

Listener : A named network location (e.g., port, Unix domain socket) that downstream clients can connect to. Envoy exposes one or more listeners for downstream connections.

Cluster : A set of logically similar upstream hosts that Envoy connects to. Members are discovered via service discovery and health‑checked; load‑balancing determines which member receives a request.

Mesh : A coordinated group of hosts that provides a consistent network topology. In this document, an “Envoy mesh” is a set of Envoy proxies forming the messaging foundation of a distributed system composed of many services and platforms.

Runtime Configuration : An out‑of‑band real‑time configuration system deployed with Envoy, allowing configuration changes that affect operation without restarting Envoy or modifying the main config.

Thread Model

Envoy uses a multi‑threaded architecture within a single process. One main thread handles coordination tasks, while worker threads perform listening, filtering, and forwarding. Once a connection is accepted by a listener, the rest of its lifecycle runs on a worker thread, making most of Envoy effectively single‑threaded (embarrassingly parallel) with minimal coordination code between workers. Envoy is 100 % non‑blocking; for typical workloads we recommend setting the number of worker threads equal to the machine’s hardware thread count.

Listeners

Envoy supports any number of listeners in a single process. Generally, one Envoy instance per machine is recommended regardless of the number of configured listeners, simplifying operations and statistics. Currently, Envoy only supports TCP listeners.

Each listener independently configures a stack of network‑level (L3/L4) filters. When a listener receives a new connection, the configured connection‑local filter stack is instantiated and processes subsequent events. This generic listener architecture powers most Envoy proxy tasks such as rate limiting, TLS client authentication, HTTP connection management, MongoDB sniffing, raw TCP proxying, etc.

Listeners can also be dynamically obtained via Listener Discovery Service (LDS).

Network (L3/L4) Filters

Network‑level filters form the core of Envoy’s connection handling. The filter API allows mixing and matching different filter types on a given listener. Three filter types exist:

Read: Invoked when Envoy receives data from a downstream connection.

Write: Invoked when Envoy sends data to a downstream connection.

Read/Write: Invoked for both receiving from and sending to downstream connections.

The API operates on raw bytes and a few connection events (e.g., TLS handshake completion, local or remote disconnect). Filters in the chain can pause and later resume processing, enabling advanced scenarios such as calling external rate‑limit services. Envoy ships with many built‑in network filters, documented in the architecture overview and configuration reference.

HTTP Connection Manager

HTTP is a key component of modern service‑oriented architectures, and Envoy implements extensive HTTP functionality via a built‑in network filter called the HTTP Connection Manager. This filter converts raw bytes into HTTP‑level messages and events (headers, bodies, trailers), handles connection and access logging, request ID generation, header processing, routing table management, and statistics.

The HTTP Connection Manager natively supports HTTP/1.1, WebSockets, and HTTP/2 (but not SPDY). It is designed primarily as an HTTP/2 multiplexing proxy. Internally, HTTP terminology such as “stream” is used to describe request/response flows. Codec APIs translate protocol‑specific features (e.g., HTTP/1.1 pipelining) into a protocol‑agnostic form, so most code does not need to know whether a stream originated from HTTP/1.1 or HTTP/2.

HTTP Header Sanitization

For security reasons, the HTTP Connection Manager performs various header sanitization operations.

Route Configuration

Each HTTP Connection Manager filter has an associated route table, which can be specified either statically or dynamically via the RDS API.