Optimizing Data Synchronization for Xianyu IM: Layered Architecture and Performance Improvements

Background: Rapid user growth has stressed Xianyu's instant messaging (IM) system. The existing client‑side code became tangled, leading to hidden data‑sync issues that magnify with scale.

The backend partitions data packets into domains, each with a unique, sequential version number. Clients record the latest version per domain and request incremental updates based on those numbers.

To handle offline users, a push‑pull hybrid is used: ACCS pushes live updates, while offline clients pull missing packets.

Analysis: The current sync strategy works but suffers from: (1) frequent intensive pushes causing redundant domain syncs and wasted network/CPU resources; (2) server blind to client consumption status; (3) unclear separation of parsing, storage, and business logic, hindering targeted A/B testing.

The solution is to extract a dedicated data‑sync layer, making the process reusable for other services.

Data Sync Optimization – Split & Layer: The server attaches domain info to each packet and pushes it through the sync layer. The client uses domain info to route packets to the appropriate consumer, decoupling business protocols from transport.

Layered Model: Steps include establishing an ACCS long‑link on app start, registering consumers for specific domains, buffering incoming packets per domain, prioritizing packets, blocking reads during domain sync, and acknowledging consumed packets back to the server.

Domain Sync Protocol: Each request carries target user ID, domain ID with priority, and current version.

Sorting Strategy: Packets are stored in a version‑keyed map, achieving O(log n) operations and allowing newer packets to overwrite stale ones.

Issues & Solutions: Simultaneous push and pull can cause duplicate delivery and traffic waste. Introducing a middle‑layer buffers incoming packets while a domain sync is in progress, merges results, then resumes processing.

Domain Priority: Chat messages have higher priority than promotional messages. Separate queues per priority ensure high‑priority packets are consumed first, and priority can be adjusted dynamically.

Optimization Effects: In stress tests (100 out‑of‑order packets within 500 ms), processing latency dropped by 31% and traffic consumption decreased by 35%.

Future Plans: Dynamically adjust sync priority per scenario, enforce exclusive push or pull modes, and further monitor and auto‑repair message integrity.