Design and Implementation of a High‑Performance Spring WebFlux API Gateway (ship‑gate)

Background

Inspired by the Soul gateway on GitHub, the author built a high‑performance gateway named ship‑gate over two weeks, completing core functionality but lacking a management UI due to limited front‑end skills.

Design

2.1 Technology Selection

The gateway must handle massive traffic, so asynchronous request handling is required. Two common stacks are considered:

Tomcat/Jetty + NIO + Servlet 3 (asynchronous support, used by JD, Youzan, Zuul)

Netty + NIO (high‑concurrency, used by Vipshop with >300k TPS)

Spring WebFlux (built on Netty) is chosen to avoid manual HTTP handling.

Additional requirements include extensibility via a filter chain, service discovery/registration (Nacos is selected over Zookeeper), custom routing, high availability, gray release, authentication, and load‑balancing.

2.2 Requirements List

Key features:

Custom routing rules (DEFAUL, HEADER, QUERY) with operators =, regex, like

Cross‑language support (HTTP)

High performance (Netty + in‑JVM caching)

High availability (stateless cluster mode)

Gray release (canary/A‑B testing)

Interface authentication (plug‑in based)

Load‑balancing (random, round‑robin, weighted, SPI configurable)

2.3 Architecture Design

The project is split into modules similar to Zuul, Spring Cloud Gateway, and Soul. (Diagram omitted)

2.4 Database Schema

(Diagram omitted)

Implementation

3.1 ship‑client‑spring‑boot‑starter

A Spring Boot starter registers the service to Nacos and notifies ship‑admin on startup/shutdown.

public class AutoRegisterListener implements ApplicationListener<ContextRefreshedEvent> {
    private static final Logger LOGGER = LoggerFactory.getLogger(AutoRegisterListener.class);
    private volatile AtomicBoolean registered = new AtomicBoolean(false);
    private final ClientConfigProperties properties;
    @NacosInjected private NamingService namingService;
    @Autowired private RequestMappingHandlerMapping handlerMapping;
    private final ExecutorService pool;
    private static List<String> ignoreUrlList = new LinkedList<>();
    static { ignoreUrlList.add("/error"); }
    public AutoRegisterListener(ClientConfigProperties properties) { /* ... */ }
    private boolean check(ClientConfigProperties properties) { /* ... */ }
    @Override public void onApplicationEvent(ContextRefreshedEvent event) { /* register and hook */ }
    private void registerShutDownHook() { /* send unregister request */ }
    private void doRegister() { /* register instance to Nacos and ship‑admin */ }
    private RegisterAppDTO buildRegisterAppDTO(Instance instance) { /* ... */ }
}

3.2 ship‑server

The server intercepts requests with a WebFilter, builds a plugin chain, and forwards the request to the selected backend instance.

public class PluginFilter implements WebFilter {
    private ServerConfigProperties properties;
    public PluginFilter(ServerConfigProperties properties) { this.properties = properties; }
    @Override public Mono<Void> filter(ServerWebExchange exchange, WebFilterChain chain) {
        String appName = parseAppName(exchange);
        if (CollectionUtils.isEmpty(ServiceCache.getAllInstances(appName))) {
            throw new ShipException(ShipExceptionEnum.SERVICE_NOT_FIND);
        }
        PluginChain pluginChain = new PluginChain(properties, appName);
        pluginChain.addPlugin(new DynamicRoutePlugin(properties));
        pluginChain.addPlugin(new AuthPlugin(properties));
        return pluginChain.execute(exchange, pluginChain);
    }
    private String parseAppName(ServerWebExchange exchange) { /* ... */ }
}

The PluginChain holds enabled plugins and executes them sequentially.

public class PluginChain extends AbstractShipPlugin {
    private int pos;
    private List<ShipPlugin> plugins;
    private final String appName;
    public PluginChain(ServerConfigProperties properties, String appName) { super(properties); this.appName = appName; }
    public void addPlugin(ShipPlugin shipPlugin) { /* filter by enabled, sort by order */ }
    @Override public Mono<Void> execute(ServerWebExchange exchange, PluginChain pluginChain) {
        if (pos == plugins.size()) return exchange.getResponse().setComplete();
        return pluginChain.plugins.get(pos++).execute(exchange, pluginChain);
    }
}

DynamicRoutePlugin

performs routing based on version matching and load‑balancing.

public class DynamicRoutePlugin extends AbstractShipPlugin {
    private static final Logger LOGGER = LoggerFactory.getLogger(DynamicRoutePlugin.class);
    private static WebClient webClient;
    static { /* configure HttpClient with timeouts */ }
    @Override public Integer order() { return ShipPluginEnum.DYNAMIC_ROUTE.getOrder(); }
    @Override public String name() { return ShipPluginEnum.DYNAMIC_ROUTE.getName(); }
    @Override public Mono<Void> execute(ServerWebExchange exchange, PluginChain pluginChain) {
        String appName = pluginChain.getAppName();
        ServiceInstance instance = chooseInstance(appName, exchange.getRequest());
        String url = buildUrl(exchange, instance);
        return forward(exchange, url);
    }
    private Mono<Void> forward(ServerWebExchange exchange, String url) { /* proxy request */ }
    private boolean requireHttpBody(HttpMethod method) { /* POST/PUT/PATCH */ }
    private String buildUrl(ServerWebExchange exchange, ServiceInstance instance) { /* construct target URL */ }
    private ServiceInstance chooseInstance(String appName, ServerHttpRequest request) { /* version match + load‑balance */ }
    private String matchAppVersion(String appName, ServerHttpRequest request) { /* rule evaluation */ }
    private boolean match(AppRuleDTO rule, ServerHttpRequest request) { /* HEADER/QUERY/DEFAULT */ }
}

3.3 Data Synchronization

Two scheduled tasks keep Nacos metadata and local caches in sync.

public class NacosSyncListener implements ApplicationListener<ContextRefreshedEvent> {
    private static final Logger LOGGER = LoggerFactory.getLogger(NacosSyncListener.class);
    private static ScheduledThreadPoolExecutor scheduledPool = new ScheduledThreadPoolExecutor(1, new ShipThreadFactory("nacos-sync", true).create());
    @NacosInjected private NamingService namingService;
    @Value("${nacos.discovery.server-addr}") private String baseUrl;
    @Resource private AppService appService;
    @Override public void onApplicationEvent(ContextRefreshedEvent event) { /* schedule NacosSyncTask */ }
    class NacosSyncTask implements Runnable { /* push weight & plugins to Nacos */ }
}

Local cache refresh reads all instances from Nacos and updates ServiceCache and PluginCache.

public class DataSyncTaskListener implements ApplicationListener<ContextRefreshedEvent> {
    private static ScheduledThreadPoolExecutor scheduledPool = new ScheduledThreadPoolExecutor(1, new ShipThreadFactory("service-sync", true).create());
    @NacosInjected private NamingService namingService;
    @Autowired private ServerConfigProperties properties;
    @Override public void onApplicationEvent(ContextRefreshedEvent event) { /* schedule DataSyncTask */ }
    class DataSyncTask implements Runnable { /* pull instances, build ServiceInstance list, update caches */ }
}

3.4 WebSocket Cache Synchronization

The server pushes routing rule changes to clients via a WebSocket server.

public class WebsocketSyncCacheServer extends WebSocketServer {
    private static final Logger LOGGER = LoggerFactory.getLogger(WebsocketSyncCacheServer.class);
    private Gson gson = new GsonBuilder().create();
    private MessageHandler messageHandler = new MessageHandler();
    public WebsocketSyncCacheServer(Integer port) { super(new InetSocketAddress(port)); }
    @Override public void onMessage(WebSocket ws, String message) { messageHandler.handler(message); }
    class MessageHandler { public void handler(String message) { /* update RouteRuleCache */ } }
}

The client connects, receives the full rule set on open, and keeps the connection alive with reconnection logic.

public class WebsocketSyncCacheClient {
    private static final Logger LOGGER = LoggerFactory.getLogger(WebsocketSyncCacheClient.class);
    private WebSocketClient client;
    private RuleService ruleService;
    private Gson gson = new GsonBuilder().create();
    public WebsocketSyncCacheClient(@Value("${ship.server-web-socket-url}") String url, RuleService ruleService) { /* connect and schedule reconnection */ }
    public <T> void send(T t) { while (!client.getReadyState().equals(ReadyState.OPEN)) { LOGGER.debug("connecting ...please wait"); } client.send(gson.toJson(t)); }
}

Testing

4.1 Dynamic Routing Test

Two instances (gray_1.0 and prod_1.0) are registered in Nacos. A header rule name=ship routes traffic to the gray version. Using Postman with the header hits only the gray instance, verified by logs.

4.2 Performance Benchmark

Running wrk on a MacBook Pro (2.3 GHz i7, 16 GB) with 20 threads and 500 connections yields ~9,400 requests per second, demonstrating the gateway’s high throughput.

Conclusion

The project shows that building a high‑performance gateway is achievable with Spring WebFlux, Netty, and Nacos. The author encountered and resolved several issues, contributed to upstream projects, and provides the full source at https://github.com/2YSP/ship-gate.