Real-Time Loss Prevention System: Architecture and Implementation at YouZan

This article introduces YouZan's real-time loss prevention and verification system, designed to detect financial discrepancies in e-commerce transactions. As business volume grows and scenarios become more complex, the need to quickly identify issues and prevent financial losses became critical.

System Evolution:

The platform evolved through two versions: the first SQL-based approach had low entry barriers but suffered from performance issues, inability to handle large fields, and poor awareness of DDL/DML changes. The second hard-coded version solved some problems but introduced tight coupling with business logic and reduced maintainability.

Current Architecture (5 Layers):

Data Source Layer: Monitors binlog from various system databases

Data Collection Layer: Parses and filters binlog messages using Groovy scripts and SpEL expressions

Data Transformation Layer: Maps and transforms data into abstract models using default field mapping or Groovy scripts

Verification Layer: Performs reconciliation using default amount comparison or custom Groovy scripts

Exception Handling Layer: Triggers alerts and preserves verification snapshots for investigation

Key Technical Features:

Processes 200 million binlog messages daily

Handles over 60 million verification checks daily

NSQ QPS peak reaches 12,000

Sharding strategy with dynamic scaling capabilities

Redis + TMC local cache for hot keys

Distributed lock for sequential processing

Use Cases:

Payment vs. settlement reconciliation (detecting over/under charging)

Refund verification (duplicate payment detection)

System migration verification (dual-write consistency)

DML modification detection (manual data changes)