How Does Envoy Proxy Process a Request? A Deep Dive into Its Lifecycle

Terminology

Envoy uses the following terms:

Cluster : a logical service consisting of a set of endpoints to which Envoy forwards requests.

Downstream : the entity that connects to Envoy, either a local sidecar application or a remote client.

Endpoints : network nodes that implement a logical service; they form clusters.

Filter : a module in the connection or request processing pipeline that provides specific functionality, similar to Unix filters.

Filter chain : a series of filters.

Listeners : Envoy modules that bind to an IP/port, accept new TCP connections (or UDP packets), and orchestrate downstream requests.

Upstream : the endpoint to which Envoy forwards a request, either a local application or a remote node.

Network topology

Envoy originated as a sidecar proxy in a service mesh, handling load balancing, routing, observability, security, and service discovery. In a mesh, requests may pass through an Envoy acting as a gateway or via ingress/egress listeners.

Ingress listeners receive requests from other nodes and forward them to a local application; the application's response returns through Envoy.

Egress listeners receive requests from a local application and forward them to other nodes, which typically also run Envoy.

Configuration

Envoy is highly extensible. A request path can be composed from the following building blocks:

L3/4 protocols: TCP, UDP, Unix domain sockets

L7 protocols: HTTP/1, HTTP/2, HTTP/3, gRPC, Thrift, Dubbo, Kafka, Redis, various databases

Transport sockets: cleartext, TLS, ALTS

Connection routing: PROXY protocol, source address, dynamic forwarding

Circuit breakers and outlier detection

Network‑related configuration: HTTP listeners, access logs, health checks, tracing, statistics extensions

The example below demonstrates a static bootstrap configuration that defines a single listener and a single cluster, with static routing and static endpoints.

static_resources:
  listeners:
    # A single listener bound to port 443.
    - name: listener_https
      address:
        socket_address:
          protocol: TCP
          address: 0.0.0.0
          port_value: 443
      listener_filters:
        - name: "envoy.filters.listener.tls_inspector"
          typed_config: {}
      filter_chains:
        - filter_chain_match:
            server_names: ["acme.com"]
          transport_socket:
            name: envoy.transport_sockets.tls
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
              common_tls_context:
                tls_certificates:
                  - certificate_chain: { filename: "certs/servercert.pem" }
                private_key: { filename: "certs/serverkey.pem" }
          filters:
            - name: envoy.filters.network.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                stat_prefix: ingress_http
                use_remote_address: true
                http2_protocol_options:
                  max_concurrent_streams: 100
                access_log:
                  - name: envoy.access_loggers.file
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
                      path: "/var/log/envoy/access.log"
                route_config:
                  name: local_route
                  virtual_hosts:
                    - name: local_service
                      domains: ["acme.com"]
                      routes:
                        - match:
                            path: "/foo"
                          route:
                            cluster: some_service
                http_filters:
                  - name: some.customer.filter
                  - name: envoy.filters.http.router
  clusters:
    - name: some_service
      connect_timeout: 5s
      transport_socket:
        name: envoy.transport_sockets.tls
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
      load_assignment:
        cluster_name: some_service
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: 10.1.2.10
                      port_value: 10002
              - endpoint:
                  address:
                    socket_address:
                      address: 10.1.2.11
                      port_value: 10002
      http2_protocol_options:
        max_concurrent_streams: 100
      stats_sinks:
        - name: envoy.stat_sinks.statsd
          typed_config:
            "@type": type.googleapis.com/envoy.config.metrics.v3.StatsdSink
            tcp_cluster_name: some_statsd_cluster

High‑level architecture

Envoy’s request handling consists of two major subsystems:

Listener subsystem : processes downstream requests, manages their lifecycle, and includes downstream HTTP/2 codecs.

Cluster subsystem : selects and configures connections to upstream endpoints, performs health checking, load balancing, and connection pooling, and includes upstream HTTP/2 codecs.

Both subsystems are bridged by the HTTP router filter, which forwards downstream HTTP requests to the appropriate upstream cluster.

Request flow

Overview

The Envoy listener running on a worker thread accepts a downstream TCP connection.

A listener filter chain is created and runs; it extracts SNI and other TLS‑pre‑handshake information, then matches a network filter chain.

The TLS transport socket decrypts data from the TCP connection.

A network filter chain runs; the HTTP connection manager (HCM) is the final network filter.

The HCM’s HTTP/2 codec frames the decrypted stream into independent HTTP/2 streams.

For each HTTP stream, an HTTP filter chain is created. A custom filter may modify the request before the router filter selects a route and cluster.

The chosen cluster performs load balancing to pick an endpoint; if no connection exists, a new one is created.

The upstream endpoint’s HTTP/2 codec multiplexes the request onto the upstream connection.

The upstream TLS transport socket encrypts the bytes and writes them to the upstream TCP socket.

The response travels back through the same filter chain in reverse order, ending with the downstream response.

When the response is complete, the stream is destroyed, statistics are updated, access logs are written, and tracing spans are finalized.

Listener TCP connection reception

The ListenerManager instantiates listeners based on configuration. Listeners can be in three states: Warming (waiting for dependencies), Active (bound and accepting connections), and Draining (no longer accepting new connections but serving existing ones). Workers may share a listening socket via SO_REUSEPORT , and the kernel decides which worker thread accepts a new connection.

Listener and network filter chain matching

After a TCP connection is accepted, a listener filter chain runs. In TLS mode, the envoy.filters.listener.tls_inspector extracts the SNI and matches the appropriate network filter chain. The filter chain may be inserted automatically when SNI or ALPN is required.

TLS transport socket decryption

Envoy’s TransportSocket interface implements TLS decryption. When data is ready, Network::ConnectionImpl::onReadReady() invokes SslSocket::doRead(), which performs the TLS handshake and then provides a decrypted byte stream to the Network::FilterManagerImpl.

Network filter chain processing

Envoy creates a series of network filters for each new connection. Filters implement ReadFilter, WriteFilter, or both. The final network filter is the HTTP connection manager, which creates the HTTP/2 codec and hands off processing to the HTTP filter chain.

HTTP/2 codec decoding

The HTTP/2 codec (based on nghttp2) converts the byte stream into HTTP/2 frames, multiplexing multiple independent streams over a single connection. Each stream carries its own request and response headers, body, and trailers.

HTTP filter chain processing

For each HTTP stream, the HCM instantiates an HTTP filter chain. Filters can be StreamDecoderFilter, StreamEncoderFilter, or both. The custom filter runs first, possibly modifying headers, then the router filter selects a route and the corresponding upstream cluster.

Load balancing

Each cluster has a load‑balancing policy (round‑robin, Maglev, least‑request, random, etc.) that selects an endpoint. The selection may use static configuration, DNS, xDS, or health‑check data. If no connection exists or the pool is exhausted, a new connection is created unless a circuit breaker prevents it.

Upstream request handling

The router filter obtains an HTTP connection pool from the ClusterManager . It creates an UpstreamRequest, which uses the pool’s connection to send encoded request headers via UpstreamRequest::encoderHeaders(). The upstream endpoint’s HTTP/2 codec multiplexes the request, and its TLS transport socket encrypts the data before writing to the TCP socket.

Response path and HTTP lifecycle

The response traverses the same filter chain in reverse order. HTTP filters invoke encoder callbacks, and network filters handle read/write events as data flows in both directions. When the upstream response reaches the end of the stream, the router filter finalizes the request.

Post‑request processing

Statistics are updated (some are flushed periodically by the main thread).

Access logs are written to the configured file logger.

Trace spans are generated if tracing is enabled.