How to Build a Scalable WebSocket Communication Service with Reliable Messaging

Background

The existing system had many backend services tightly coupled with frontend web interactions, using polling for asynchronous results and ad‑hoc WebSocket integrations, which caused high maintenance cost and low responsiveness.

Polling was used for payment callbacks and business order results.

Some features required instant push (e.g., QR code refresh, charging port updates).

Synchronously called services needed to be refactored to asynchronous calls with WebSocket push to reduce latency.

Future business scenarios will also need real‑time communication.

Goals

Standardize WebSocket project structure and coding conventions.

Decouple business code to improve integration efficiency.

Replace unreasonable polling with push notifications.

Transform synchronous calls into asynchronous ones, pushing results via WebSocket to improve overall response speed.

Replace Redis pub/sub with a message queue (MQ) for reliable communication.

Core Design

The service abstracts WebSocket (ws) as an internal communication layer, applying Domain‑Driven Design (DDD) principles to model messaging domains and reduce coupling.

Project Structure

Business Flow

3.1 Application Relationship Diagram

3.2 Business Sequence Diagram

WebSocket client registers and establishes a connection.

Server pushes messages to the client.

Client receives and processes the messages.

Ensuring Reliable Message Transmission

Three typical loss scenarios are considered:

Message loss during producer to RabbitMQ server transmission.

Server crash before persisting the message.

Consumer crashes after receiving the message but before processing it.

Producer‑Side Reliability

Two mechanisms are recommended:

Transactional mode – enable RabbitMQ transaction, roll back on failure using channel.txRollback() and retry.

Publisher confirm – enable confirm mode; the broker sends an ack on success or nack on failure, allowing the producer to retry.

Confirm mode is asynchronous and offers higher throughput, so it is generally preferred.

Broker‑Side Reliability

Enable message persistence so that messages are written to disk and survive broker restarts.

Consumer‑Side Reliability

Disable automatic acknowledgments and use manual consumer ack after business logic completes. If processing fails, the message is not acked and will be redelivered.

Message Classification

5.1 Client → Server

Scenario: client pushes a request to the server (e.g., QR code refresh trigger).

5.2 Server → Client

Scenario: server triggers an event and pushes the result to a client (e.g., payment completion callback).

5.3 Client → Client

Scenario: one client pushes a message to another client (e.g., user‑initiated request displayed on an H5 page).

WebSocket API Design

6.1 Request WebSocket Connection Token

Method: GET

Endpoint:

https://domain/xxx/websocket/token

{
  "result": 0,
  "description": "无",
  "data": "wss://dws.test.com:8086/socket/asrwqgvsd" // connection URL
}

6.2 Connect Using the Returned URL

Connection scheme: wss

URL format:

wss://dws.test.com:8086/socket/{token}

Unified Message Payload

{
  "sendType":"",
  "messageType":"Message Type",
  "businessType":"",
  "fromUniqueId":"Sender Unique ID",
  "toUniqueId":"Receiver Unique ID",
  "fromClientType":"Sender Client Type",
  "toClientType":"Receiver Client Type",
  "timestamp":0,
  "content":"Business Data",
  "tags":["Tag Set"],
  "businesses":["Business Set"]
}

Unified Call Method

The WebSocket API is encapsulated into a single client library that hides connection handling, token acquisition, and message serialization, allowing business modules to invoke real‑time communication with a single method call.

Conclusion

The design abstracts WebSocket into a reusable communication service, applies DDD modeling to lower coupling, and provides a unified API and message format. This enables any business module to adopt real‑time messaging instantly without dealing with low‑level WebSocket configuration, while ensuring reliability through transactional, confirm, persistence, and manual‑ack mechanisms.