ZZ Business Check Platform: System Design and Implementation for Real-Time Data Validation

Background – As the ZhaiZhai (Zhuanzhuan) business rapidly expands, its topology becomes increasingly complex with numerous transaction models (C2C, C2B, B2B, B2C, C2B2C). This complexity leads to RPC failures, message loss, bugs, and rule conflicts that cause data anomalies, user complaints, and financial loss. No unified data‑validation governance solution existed, prompting the creation of the ZZ Business Check Platform (ZZBCP) to provide real‑time online data verification.

System Goals – The platform aims to (1) discover abnormal online data instantly and notify relevant personnel, reducing impact on users and business; (2) enable low‑cost integration of diverse data‑validation scenarios via configurable backend rules; and (3) collect monitoring data in real time, generating statistical reports to track data quality.

System Design Overview – The execution flow starts with a trigger message that drives rule execution (immediate or delayed). The target data source, if not pre‑configured, is fetched via RPC. The target data is then validated against defined rules.

3.1 Execution Sequence Diagram – At time T1 the system receives a trigger message, matches rules, and schedules validation for T3. At T2 the target message arrives, is processed and temporarily stored in Codis, and later validated at T3, after which subsequent actions are taken based on the result.

3.2 Message Subscription Module – Currently supports MQ and Binlog subscriptions (Redis/ES not yet). It normalizes trigger and target events into a standard ZZBCP format. Binlog data from MySQL/TiDB is captured via CDC middleware (Canal, Maxwell) and pushed to Kafka.

3.3 Rule Execution Module – When a delayed rule expires, the system assembles data from Redis (target source) into the required format and hands it to the rule engine.

3.4 Rule Engines – Two engines are provided: (a) Aviator engine for simple, high‑efficiency expressions, with built‑in functions for Redis and common middleware; (b) Java engine for complex logic requiring RPC‑fetched target data, supporting dynamic compilation via Java’s native compiler and careful ClassLoader handling.

3.5 Alert Module – Based on rule‑engine results, the module decides whether to trigger alerts, collect abnormal data, or retry validation. Alert SMS content can be customized to expose necessary business fields for troubleshooting.

3.6 Automatic Data Repair Module – For specific abnormal scenarios, the platform can automatically trigger data repair actions (e.g., internal financial reconciliation system). Repair logic should remain within the business domain; ZZBCP only provides the trigger entry point.

3.7 Monitoring Metrics – All rule information is reported to ZhaiZhai’s Prometheus‑based monitoring system, which aggregates metrics such as rule hits, successful validations, and failures.

3.8 Backend Configuration – The admin UI allows registration of events, alarm settings, gray‑release controls, and manual rule execution, enabling rapid configuration and testing of validation logic.