Overview of CDC Tools: Canal, Maxwell, Databus, and Alibaba DTS

Author: stone-no1 (Source: lian jblog.csdn.net/weixin_38071106/article/details/88547660)

Canal

Positioning: a MySQL‑based incremental log parser that provides data subscription and consumption.

Principle: Canal simulates a MySQL slave, sends a dump request to the master, receives binary logs, and parses the byte stream.

The parser workflow includes obtaining the last processed binlog position, establishing a connection with BINLOG_DUMP, receiving binary logs, parsing them, storing events via EventSink, and periodically recording the binlog position.

Additional capabilities: data filtering (wildcards, tables, fields), routing/distribution (1:n), merging (n:1), and preprocessing (e.g., joins) before storage.

Maxwell

Canal is Java‑based with server and client components; it requires custom client development to consume parsed data.

Maxwell simplifies this by outputting data changes as JSON strings, eliminating the need for custom client code.

Databus

Databus is a low‑latency change‑capture system used in LinkedIn’s data pipelines, providing source‑consumer isolation, ordered at‑least‑once delivery, bootstrapping from any point, partitioned consumption, and source‑consistent storage.

Alibaba Data Transmission Service (DTS)

DTS is Alibaba Cloud’s data flow service supporting RDBMS, NoSQL, and OLAP sources, offering migration, real‑time subscription, and synchronization without downtime, suitable for disaster recovery, multi‑active regions, cross‑border sync, data warehousing, and cache updates.

Advantages include high performance, security, rich features, and seamless integration with Alibaba RDS and DRDS, handling binlog retention, primary‑secondary switch, and VPC changes.

In practice, DTS acts like a message queue delivering wrapped SQL objects that can be parsed by custom services.