Using Flink CDC to Capture MySQL Changes and Sink Them into ClickHouse

Change Data Capture (CDC) is introduced as a technique to capture INSERT, UPDATE, DELETE operations from databases and forward them downstream, with typical scenarios such as heterogeneous database synchronization, microservice state sharing, and cache or CQRS view updates.

A comparison table highlights the differences between query‑based CDC (batch, full‑table scans) and log‑based CDC (streaming, binlog monitoring) and lists open‑source tools like Sqoop, Kafka JDBC Source, Canal, Maxwell, and Debezium.

Debezium is described as an open‑source, low‑latency streaming platform that monitors database logs, provides a unified change event model, and ensures durability and exactly‑once processing.

ClickHouse is presented as a column‑oriented, real‑time analytical database with advantages such as columnar storage, data compression, distributed processing, and SQL support, as well as disadvantages like lack of full transaction support and limited UPDATE/DELETE capabilities.

Flink CDC example code shows how to create a MySQL source using MySQLSource.builder(), a custom JsonDebeziumDeserializationSchema to convert Debezium records to JSON, and a ClickhouseSink that writes the JSON data into ClickHouse via JDBC. The full Java source, deserialization schema, and sink implementation are provided within ... blocks.

Flink SQL CDC is demonstrated with three SQL statements: a source table DDL using the mysql-cdc connector, a sink table DDL using the jdbc connector, and a transformation SQL that inserts data from the source to the sink.

The required Maven dependencies for Flink core, streaming, table API, JDBC connector, MySQL CDC connector, ClickHouse connector, and Gson are listed.

Reference links to external articles and tutorials are included for further reading.