How to Build a Distributed WebSocket Cluster with Spring Cloud and Consistent Hashing

Problem Background

During a recent project the author needed multi‑user communication via WebSocket, which required handling the WebSocket handshake and sharing WebSocket sessions across a cluster.

Scenario Description

Resources : 4 servers – one with SSL domain, one Redis+MySQL server, two application servers (cluster).

Deployment Constraint : Only the SSL‑enabled server can act as the API gateway, handling HTTPS and WSS requests.

Requirement : Users must log in, establish a WSS connection, and be able to send both one‑to‑one and group messages.

Cluster Service Type : Each instance serves both stateless HTTP requests and long‑lived WebSocket connections.

Technology Stack

Eureka service discovery

Redis session sharing

Redis pub/sub

Spring Boot

Zuul gateway

Spring Cloud Gateway

Spring WebSocket

Ribbon load balancing

Netty NIO framework

Consistent Hash algorithm

Technical Feasibility Analysis

WebSocketSession cannot be serialized to Redis, so true WebSocket session sharing is impossible; only HttpSession can be shared via spring-session-data-redis and spring-boot-starter-redis.

Solution Evolution

Netty vs Spring WebSocket

Netty offers high concurrency but integrates poorly with Spring Cloud, requires duplicate business logic, and lacks easy REST support.

Spring WebSocket is fully integrated with Spring Boot, requiring only a few configuration steps:

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-websocket</artifactId>
</dependency>

@Configuration
public class WebSocketConfig implements WebSocketConfigurer {
    @Override
    public void registerWebSocketHandlers(WebSocketHandlerRegistry registry) {
        registry.addHandler(myHandler(), "/").setAllowedOrigins("*");
    }
    @Bean
    public WebSocketHandler myHandler() {
        return new MessageHandler();
    }
}

@Component
public class MessageHandler extends TextWebSocketHandler {
    private List<WebSocketSession> clients = new ArrayList<>();
    @Override
    public void afterConnectionEstablished(WebSocketSession session) {
        clients.add(session);
        System.out.println("uri :" + session.getUri());
        System.out.println("connection established: " + session.getId());
    }
    @Override
    public void afterConnectionClosed(WebSocketSession session, CloseStatus status) {
        clients.remove(session);
        System.out.println("connection closed: " + session.getId());
    }
    @Override
    protected void handleTextMessage(WebSocketSession session, TextMessage message) {
        String payload = message.getPayload();
        // broadcast to all clients
        clients.forEach(s -> {
            try { s.sendMessage(new TextMessage("server reply: " + payload)); }
            catch (Exception e) { e.printStackTrace(); }
        });
    }
}

The author chose Spring WebSocket for simplicity and compatibility with the Spring Cloud ecosystem.

From Zuul to Spring Cloud Gateway

Zuul 1.0 does not support WebSocket forwarding; Zuul 2.0 does but lacks Spring Boot integration. Therefore the gateway was switched to Spring Cloud Gateway with SSL termination and dynamic routing.

server:
  port: 443
  ssl:
    enabled: true
    key-store: classpath:xxx.jks
    key-store-password: xxxx
    key-store-type: JKS
    key-alias: alias
spring:
  application:
    name: api-gateway
  cloud:
    gateway:
      httpclient:
        ssl:
          handshake-timeout-millis: 10000
          useInsecureTrustManager: true
      discovery:
        locator:
          enabled: true
      routes:
        - id: dc
          uri: lb://dc
          predicates:
            - Path=/dc/**
        - id: wecheck
          uri: lb://wecheck
          predicates:
            - Path=/wecheck/**

@Component
public class HttpsToHttpFilter implements GlobalFilter, Ordered {
    private static final int HTTPS_TO_HTTP_FILTER_ORDER = 10099;
    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
        URI originalUri = exchange.getRequest().getURI();
        String forwardedUri = exchange.getRequest().getURI().toString();
        if (forwardedUri != null && forwardedUri.startsWith("https")) {
            URI httpUri = new URI("http", originalUri.getUserInfo(), originalUri.getHost(),
                originalUri.getPort(), originalUri.getPath(), originalUri.getQuery(), originalUri.getFragment());
            ServerHttpRequest newRequest = exchange.getRequest().mutate().uri(httpUri).build();
            ServerWebExchange newExchange = exchange.mutate().request(newRequest).build();
            return chain.filter(newExchange);
        }
        return chain.filter(exchange);
    }
    @Override
    public int getOrder() { return HTTPS_TO_HTTP_FILTER_ORDER; }
}

Session Broadcast Solution

Each gateway forwards a teacher’s broadcast request to all cluster nodes; each node checks its local sessions and sends the message to matching clients. This is simple but wastes CPU when no matching sessions exist.

Consistent Hashing Solution (Core Idea)

Use consistent hashing to map a user ID to a specific cluster node, ensuring that a user’s WebSocket session is always handled by the same server. The hash ring must handle node up/down events.

When a node goes down, its sessions are closed and the node is removed from the ring.

When a node comes up, the ring is updated and affected clients are forced to reconnect.

Node state changes are detected via Eureka listeners, and the hash ring is stored in Redis with pub/sub notifications to keep gateways in sync.

Ribbon Limitations in Finchley.RELEASE

Custom load‑balancing rules that extend AbstractLoadBalancerRule cause request mixing between different services, and Ribbon’s choose() method lacks a proper key parameter, making it difficult to implement consistent hashing directly.

Workaround: the client first makes an HTTP request containing the user ID, the gateway hashes the ID, returns the target IP, and the client then opens the WebSocket connection to that IP.

Conclusion

The article presents two viable approaches for WebSocket clustering—simple session broadcast and a more efficient consistent‑hashing routing—detailing the required Spring Cloud components, configuration snippets, and code examples.