Designing a High‑Performance Java‑Based API Gateway with Zuul2 and Netty

Background: the middleware team lacked a unified API gateway and decided to research and build a functional, highly available gateway.

Technology selection: evaluated open‑source gateways (Nginx+Lua, Java, Go, Node.js). Because the team primarily uses Java, the focus was on Zuul1, Zuul2, Spring Cloud Gateway, and Dromara Soul. After comparing architectures (Servlet+ThreadPool, NIO async, Netty), the decision was to develop the gateway on Netty and customize Zuul2.

Zuul2 Overview

Key concepts and architecture are introduced (see diagram). Zuul2 parses HTTP by streaming headers without buffering the body, meaning it does not use HttpObjectAggregator. The article shows the Netty server example and explains the roles of HttpServerCodec, HttpObjectAggregator, HttpRequestDecoder, and HttpResponseEncoder.

HTTP Parsing in Zuul2

Zuul2’s source adds HttpServerCodec and connection handlers to the Netty pipeline, avoiding body aggregation and handling chunked transfer manually.

Data Flow

Netty converts raw bytes to HttpRequest, then ClientRequestReceiver creates a Zuul HttpRequestMessageImpl. The request passes through inbound filters, the endpoint filter (ProxyEndpoint) which forwards to the backend, then outbound filters, and finally the response is written back via ClientResponseWriter. Zuul2 processes request headers and body in separate filter passes.

Filter Chains

Two chains exist: the Netty ChannelPipeline chain and the Zuul filter chain (inbound, endpoint, outbound). A GroovyFilterFileManagerPoller periodically scans a directory to hot‑reload filters.

Gateway Features

Service registration & discovery via Zookeeper SDK or manual configuration.

Dynamic routing: nearby routing and gray/tag routing.

Load balancing (weighted random, round‑robin, consistent hash) implemented similarly to Dubbo.

Dynamic configuration via HTTP push, pull, and incremental pull.

Protocol conversion (HTTP↔HTTP, HTTP↔Dubbo, HTTP↔Tars).

Security mechanisms: IP whitelist/blacklist, OAuth, appKey/appSecret verification, encryption, Vivo login validation.

Monitoring & alerting of QPS, latency, and node resources.

Rate limiting & circuit breaking integrated with Hystrix/Sentinel.

Zero‑downtime deployment using an online flag and OnlineRouter to filter offline nodes.

Cluster isolation by grouping business services.

System Architecture

Module interaction diagram and management console layout are provided (images).

Implementation Details

Dynamic configuration uses Netflix Archaius; a custom GatewayConfigConfiguration extends ConcurrentMapConfiguration (code snippet). Guice controls bean loading order.

Routing mimics Dubbo routing: the gateway extracts a service name from the path prefix or request header, matches APIs, applies load balancing, and supports gray routing based on a header value.

Load‑balance interface ILoadBalance and related implementations are shown.

Health‑check uses a scheduled executor with HttpAsyncClient (dependency snippet).

Logging is switched from log4j to Logback’s AsyncAppender for asynchronous logging.

Protocol conversion implementations: HTTP↔Dubbo via generic invoke, HTTP↔Tars via the TarsGateway project.

Zero‑downtime release adds an online field to dynamic_forward_server and an OnlineRouter that filters out offline machines (code snippet).

Cluster grouping isolates traffic between different business lines; new gateway groups can be added without deploying separate front‑ends.

Migration Guidance

When existing services use Spring Cloud Gateway without clear path prefixes, a "microservice set" to service‑list mapping is introduced. Clients send an scTag header to indicate the microservice set, allowing the gateway to resolve the appropriate backend without changing client code.