Transform Envoy Access Logs: Custom Log Formats for Full Observability

Many observability problems stem not from system design flaws but from using Envoy's default access log format, which is sparse and unsuitable for modern micro‑service architectures.

Fortunately, Envoy offers flexible custom access log capabilities, supporting plain text, JSON output, dynamic variable substitution, and even color highlighting. A single log_format line can turn cryptic logs into useful reports.

Default Envoy Access Log

When you first enable Envoy file access logging, you may see output like this:

[2025-09-16T09:15:06.113Z] "GET /version HTTP/1.1" 200 - 0 99 1 1 "172.17.0.1" "curl/7.29.0" "817b0995-8552-46be-9151-8f0a8ebf6fce" "localhost:10000" "172.139.20.170:8090"

Although it contains basic request method, path, status code, and latency, it lacks critical information:

No real client IP (X‑Forwarded‑For)

No request unique identifier (X‑Request‑ID)

No user‑agent

No response body size

Fields are poorly delimited, making machine parsing difficult

This cannot support fine‑grained monitoring, auditing, or troubleshooting.

Custom Envoy Access Log

Envoy’s access_log section supports highly customizable output. Below is a YAML snippet that configures JSON‑formatted logs with a rich set of fields:

static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address:
        address: 0.0.0.0
        port_value: 10000
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: ingress_http
          # Console log output
          access_log:
          - name: envoy.access_loggers.stdout
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
              log_format:
                json_format:
                  authority_for: "%REQ(:AUTHORITY)%"
                  bytes_received: "%BYTES_RECEIVED%"
                  bytes_sent: "%BYTES_SENT%"
                  downstream_local_address: "%DOWNSTREAM_LOCAL_ADDRESS%"
                  downstream_remote_address: "%DOWNSTREAM_REMOTE_ADDRESS%"
                  duration: "%DURATION%"
                  istio_policy_status: "%DYNAMIC_METADATA(istio.mixer:status)%"
                  method: "%REQ(:METHOD)%"
                  path: "%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%"
                  protocol: "%PROTOCOL%"
                  request_id: "%REQ(X-REQUEST-ID)%"
                  requested_server_name: "%REQUESTED_SERVER_NAME%"
                  response_code: "%RESPONSE_CODE%"
                  response_flags: "%RESPONSE_FLAGS%"
                  route_name: "%ROUTE_NAME%"
                  start_time: "%START_TIME%"
                  trace_id: "%REQ(X-B3-TRACEID)%"
                  upstream_cluster: "%UPSTREAM_CLUSTER%"
                  upstream_host: "%UPSTREAM_HOST%"
                  upstream_local_address: "%UPSTREAM_LOCAL_ADDRESS%"
                  upstream_service_time: "%RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)%"
                  user_agent: "%REQ(USER-AGENT)%"
                  x_forwarded_for: "%REQ(X-FORWARDED-FOR)%"
                http_filters:
                - name: envoy.filters.http.router
                  typed_config:
                    "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
                upgrade_configs:
                - upgrade_type: websocket
                route_config:
                  name: local_route
                  virtual_hosts:
                  - name: local_service
                    domains: ["*"]
                    routes:
                    - match:
                        prefix: "/"
                      route:
                        cluster: simple_cluster
  clusters:
  - name: simple_cluster
    lb_policy: ROUND_ROBIN
    type: STATIC
    load_assignment:
      cluster_name: simple_cluster
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address: { address: 172.139.20.170, port_value: 8090 }

Testing and Verification

After applying the configuration, logs appear as structured JSON objects, for example:

{"requested_server_name":null,"istio_policy_status":null,"upstream_host":"172.139.20.170:8090","trace_id":null,"response_code":200,"protocol":"HTTP/1.1","bytes_received":0,"upstream_transport_failure_reason":null,"start_time":"2025-09-16T09:20:14.822Z","response_flags":"-","authority_for":"localhost:10000","bytes_sent":99,"duration":6,"user_agent":"curl/7.29.0","request_id":"ecb18314-3f14-4933-8ce7-bb7bb3772290","upstream_service_time":"4","authority_for":"localhost:10000","bytes_received":0,"path":"/version","downstream_local_address":"172.17.0.3:10000","downstream_remote_address":"172.17.0.1:42950","upstream_cluster":"simple_cluster","route_name":null}

Access Log Field Descriptions

authority_for ( %REQ(:AUTHORITY)%): Target hostname from the request (Host header for HTTP/1.1, :authority for HTTP/2).

bytes_received ( %BYTES_RECEIVED%): Number of bytes received in the request body (HTTP) or from downstream connection (TCP).

bytes_sent ( %BYTES_SENT%): Number of bytes sent in the response body (HTTP) or to downstream connection (TCP).

downstream_local_address ( %DOWNSTREAM_LOCAL_ADDRESS%): Destination address and port of the downstream connection.

downstream_remote_address ( %DOWNSTREAM_REMOTE_ADDRESS%): Client address and port of the downstream connection.

method ( %REQ(:METHOD)%): HTTP request method.

path ( %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%): Request path.

protocol ( %PROTOCOL%): Protocol used for the request.

request_id ( %REQ(X-REQUEST-ID)%): Unique identifier for the request.

requested_server_name ( %REQUESTED_SERVER_NAME%): Server name indicated in SSL/TLS (SNI).

response_code ( %RESPONSE_CODE%): HTTP response status code.

response_flags ( %RESPONSE_FLAGS%): Additional information about the response or connection.

route_name ( %ROUTE_NAME%): Name of the matched route; "-" if none matched.

start_time ( %START_TIME%): Timestamp when request processing started (millisecond precision).

trace_id ( %TRACE_ID%): Trace identifier for distributed tracing.

upstream_cluster ( %UPSTREAM_CLUSTER%): Upstream service cluster.

upstream_host ( %UPSTREAM_HOST%): Upstream host (typically Pod IP and port).

upstream_local_address ( %UPSTREAM_LOCAL_ADDRESS%): Local address used when establishing connection to upstream.

duration ( %DURATION%): Total time from request receipt to final byte sent to downstream (HTTP) or total connection time (TCP).

upstream_service_time ( %RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)%): Time spent processing request on upstream plus network latency.

Envoy, as the traffic entry point of modern cloud‑native architectures, provides the first window into full‑chain behavior through its access logs. Proper log_format configuration not only speeds up fault isolation but also supports security auditing, traffic analysis, and cost accounting.

Stop letting logs be a “black box”. From today, use a single configuration line to make your Envoy logs truly “speak”.