Building a Scalable, Secure Customer IM System: Architecture & Key Solutions

Introduction

In a fast‑changing business environment we needed an efficient instant‑messaging tool. Existing third‑party IM solutions lacked flexibility and customisation, so we built our own customer‑service IM system, which after three years of stable operation has greatly improved communication efficiency.

Background

Third‑party IMs were inflexible and could not meet specific business needs, limiting communication efficiency and innovation. Our self‑built system provides higher customisation and stability to support our services.

Design Philosophy

User‑Centred Interaction : Seamless experience between users, virtual assistants and human agents, with ASR converting speech to text.

Personalised Communication : Pre‑session data collection enables intent inference and tailored suggestions.

Real‑time & High Availability : Optimised response time, asynchronous processing, and high‑concurrency support.

Data Security & Privacy : Identity verification, permission control, encrypted storage and data masking.

System Architecture

The system follows a layered architecture similar to the company’s standard design:

Business Layer : Provides 24/7 online service via H5 links across apps, mini‑programs, etc.

Access Layer : Unified gateway for client connections.

Application Layer : Handles all IM business logic with dynamic extensibility.

Service Layer : Core functions ensuring stable, efficient operation.

Infrastructure Layer : Underlying infrastructure supporting all services.

Key technology choices:

Communication Protocol : WebSocket for full‑duplex, low‑latency communication.

Network Model : Java NIO for non‑blocking, multiplexed I/O.

Framework : Spring WebFlux on Netty to simplify NIO handling and enable reactive programming.

Service Centralisation

Client‑Client mode is used for peer‑to‑peer chats, while client‑server mode is adopted for customer‑service scenarios, with a central gateway routing messages.

Technical Challenges & Solutions

Challenge 1 – Gateway Design

Multi‑node deployment with a hybrid local‑plus‑Redis storage balances performance and consistency.

Challenge 2 – Session Data Storage

Cold‑hot data separation: hot sessions in Redis, cold sessions asynchronously persisted to MySQL; sharding improves read/write performance.

Challenge 3 – Message ACK

ACK mechanism confirms delivery, triggers retries on timeout, and uses unique cmId to avoid duplication.

Challenge 4 – Human Queue Mechanism

Enhanced priority rules, automatic escalation, and offline detection improve agent utilisation and user experience.

System Evolution

Continuous iteration addresses emerging business needs, enhancing agent efficiency and user satisfaction.

Benefit Analysis

Improved communication efficiency, higher user satisfaction, and stronger competitive advantage for the company.

Future Outlook

Plans include self‑service workflows with dynamic forms, tighter human‑machine collaboration, and richer data integration to further streamline support.

Conclusion

The online customer‑service IM system combines flexible architecture, data‑driven optimisation, and automation to deliver efficient communication, higher satisfaction, and sustainable growth.