Mastering Alibaba Canal: Step‑by‑Step Setup for Real‑Time MySQL Binlog Sync

Introduction

Canal is a high‑performance data synchronization system built on MySQL binary logs. It is widely used within Alibaba Group (including Taobao) to provide reliable low‑latency incremental data pipelines. The project’s source code is available on GitHub.

Canal Server parses MySQL binlog and subscribes to data changes, while Canal Client can broadcast those changes to destinations such as databases or Apache Kafka.

Key Features

Supports all platforms.

Fine‑grained system monitoring powered by Prometheus.

Multiple binlog parsing and subscription methods, including GTID.

High‑performance real‑time data sync.

HA and scalability supported by Apache ZooKeeper.

Docker support.

Drawbacks

Only incremental updates are supported; full data refresh is not.

How It Works

Canal mimics the MySQL slave protocol, pretending to be a slave and sending the handshake to the MySQL master.

The master pushes its binary log to the Canal “slave”.

Canal parses the binlog into its own data structures.

Diagram:

Preparation

2.1 Download and Extract canal‑server

Download the release package (example uses v1.1.4):

wget https://github.com/alibaba/canal/releases/download/canal-1.1.4/canal.deployer-1.1.4.tar.gz

Extract it:

tar -zxvf canal.deployer-1.1.4.tar.gz

2.2 Download and Extract canal‑adapter

Download the adapter release (example uses v1.1.4):

wget https://github.com/alibaba/canal/releases/download/canal-1.1.4/canal.adapter-1.1.4.tar.gz

Extract it:

tar -zxvf canal.adapter-1.1.4.tar.gz

Configure canal‑server

3.1 Edit canal.properties

In the conf directory, modify canal.properties as needed (e.g., set the destination name):

canal.destinations = prod  # instance name, multiple separated by commas

Create a folder for the instance and copy the example properties file:

mkdir prod
cp example/instance.properties prod/

Adjust instance.properties:

canal.instance.master.address=127.0.0.1:3306
canal.instance.dbUsername=canal
canal.instance.dbPassword=canal
canal.instance.connectionCharset=UTF-8
canal.instance.defaultDatabaseName=test_database

3.2 Start canal‑server

Enter the bin directory and launch:

cd canal-server/bin
./startup.sh &

Check the logs to confirm successful startup.

Configure canal‑adapter

4.1 Adapter Configuration

Place the MySQL 8.0 connector JAR into the lib directory:

cp mysql-connector-java-8.0.20.jar canal-adapter/lib/

Edit application.yml in the conf folder to define the source MySQL and the adapter mapping:

server:
  port: 8089
spring:
  jackson:
    date-format: yyyy-MM-dd HH:mm:ss
    time-zone: GMT+8
    default-property-inclusion: non_null
canal.conf:
  mode: tcp
  canalServerHost: 127.0.0.1:11111
  batchSize: 500
  syncBatchSize: 1000
  srcDataSources:
    defaultDS:
      url: jdbc:mysql://localhost:3306/test_database?useUnicode=true&characterEncoding=utf8&serverTimezone=Asia/Shanghai
      username: canal
      password: canal
  canalAdapters:
    - instance: prod
      groups:
        - groupId: g1
          outerAdapters:
            - name: rdb
              key: mysql1
              properties:
                jdbc.driverClassName: com.mysql.jdbc.Driver
                jdbc.url: jdbc:mysql://localhost:3306/test_database_01?useUnicode=true&characterEncoding=utf8&serverTimezone=Asia/Shanghai
                jdbc.username: canal
                jdbc.password: canal

Create a mapping file for each table under the rdb directory, for example:

dataSourceKey: defaultDS
destination: prod
outerAdapterKey: mysql1
concurrent: true
dbMapping:
  database: test_database_01
  table: test
  targetTable: test_database_01.test
  targetPk:
    id: id
  mapAll: true

4.2 Start canal‑adapter

Launch the adapter:

cd canal-adapter/bin
./startup.sh &

Check adapter.log to verify it started correctly.

Test Database Synchronization

Perform update, delete, batch insert, batch update, or batch delete operations on the source MySQL database and observe the changes being propagated through Canal to the configured destination.