How Aries Uses MySQL GTID Binlog to Power Real‑Time Data Sync at Scale

Background

Aries is a self‑developed MySQL incremental log real‑time distribution platform that has been deployed at large scale within the company. Synchronizing data between heterogeneous systems is a common challenge; ensuring stability, consistency, and the ability to trace issues after they occur motivated the design of Aries.

Implementation Principle

Aries implements the Dump protocol, masquerading as a MySQL slave to pull the binary log from the master, processes the data, and forwards it to the appropriate message middleware.

Aries is built on MySQL 5.6 GTID. GTID is globally unique within a cluster and is expressed as server_uuid:transaction_id. The server_uuid identifies the instance, while the transaction_id increments with each transaction. Using GTID avoids the forward‑rollback problems of traditional binlog file‑plus‑position replication during master‑slave switchover.

Overall Architecture

Aries consists of three major components:

Aries Ops : Handles task configuration, initialization, scheduling, and monitoring settings.

Aries Node : Executes tasks on each node, monitors node health, task status, backlog, and generates alerts.

Compared with the open‑source Canal (which follows a Server→Client model and does not support GTID), Aries adopts a Server→Message‑Queue model, supports GTID, and provides richer multi‑dimensional monitoring and alerting.

Application Scenarios

Aries is widely used in the company for MySQL‑to‑Elasticsearch synchronization, business‑logic processing, cache updates, data archiving, and real‑time statistics.

MySQL to Elasticsearch Sync : Instead of double‑writing to both MySQL and Elasticsearch (which adds complexity and risk), Aries listens to binlog events and asynchronously writes changes to Elasticsearch, ensuring eventual consistency.

Binlog in Business Processing : For logistics orders, Aries captures address and routing information after the order is placed by listening to binlog updates, enriching the data without modifying business code.

Real‑Time Statistics : By monitoring binlog streams, Aries provides instantaneous order counts per channel or partner, enabling live statistical services.

Data Archiving : For daily volumes exceeding 20 million orders, Aries archives data older than six months to HBase, using binlog‑driven deletion to filter out large transactions.

Data Sync to Big Data : Aries streams binlog data to the big‑data department for downstream analytics, ensuring timely and accurate data delivery.

Conclusion

Aries Binlog has been successfully applied across multiple business lines, effectively solving heterogeneous system synchronization and reducing code complexity. If you encounter similar scenarios, consider using a binlog‑based solution, and feel free to reach out for implementation details.