Guide to Installing and Configuring Alibaba Canal for MySQL Binlog Data Synchronization

Introduction

1.1 Canal Overview

Canal is a high‑performance data synchronization system based on MySQL binary logs, widely used within Alibaba Group (including Taobao) to provide reliable low‑latency incremental data pipelines. Its source code is available at https://github.com/alibaba/canal.

Canal Server parses MySQL binlog and publishes data changes, while Canal Client can broadcast these changes to destinations such as databases and Apache Kafka.

Key features include cross‑platform support, fine‑grained monitoring via Prometheus, GTID‑based parsing, high‑performance real‑time sync, HA/Scalability with Apache ZooKeeper, and Docker support.

Limitations: only incremental updates are supported; full‑data refresh is not.

1.2 Working Principle

The principle is simple:

Canal mimics the MySQL slave interaction protocol, disguising itself as a slave and sending a dump request to the MySQL master.

The master streams its binary logs to Canal.

Canal parses the binlog into its own data structures.

Diagram omitted.

Preparation

2.1 Download and Extract canal‑server

Download the canal‑server release (the guide uses v1.1.4) from GitHub.

root@localhost:/ # wget https://github.com/alibaba/canal/releases/download/canal-1.1.4/canal.deployer-1.1.4.tar.gz

Extract:

tar -zxvf canal.deployer-1.1.4.tar.gz

2.2 Download and Extract canal‑adapter

Similarly, download the canal‑adapter release (v1.1.4).

root@localhost:/ # wget https://github.com/alibaba/canal/releases/download/canal-1.1.4/canal.adapter-1.1.4.tar.gz

Extract:

tar -zxvf canal.adapter-1.1.4.tar.gz

Configure canal‑server

3.1 canal‑server Configuration

Edit conf/canal.properties as needed (default TCP if not using Kafka or MQ).

canal.destinations = prod # instance name, multiple separated by commas

Create a prod folder under conf and copy instance.properties from the example directory.

mkdir prod
cp example/instance.properties prod/

Modify instance.properties :

canal.instance.master.address=127.0.0.1:3306 # source MySQL address
canal.instance.dbUsername=canal # source MySQL user
canal.instance.dbPassword=canal # source MySQL password
canal.instance.connectionCharset=UTF-8 # charset must match source
canal.instance.defaultDatabaseName=test_database # default monitored database

3.2 Start canal‑server

Enter the bin directory and start:

cd canal-server/bin
./startup.sh & # background

Check logs in logs/prod to verify successful startup. Sample log excerpt shown.

Configure canal‑adapter

4.1 canal‑adapter Configuration

Place mysql-connector-java-8.0.20.jar into the lib directory.

cp mysql-connector-java-8.0.20.jar /canal-adapter/lib/

Edit conf/application.yml with server port, Kafka/RocketMQ mode, source MySQL connection, and adapter mappings.

server:
  port: 8089
spring:
  jackson:
    date-format: yyyy-MM-dd HH:mm:ss
    time-zone: GMT+8
    default-property-inclusion: non_null
canal.conf:
  mode: tcp # kafka or rocketMQ
  canalServerHost: 127.0.0.1:11111
  batchSize: 500
  syncBatchSize: 1000
srcDataSources:
  defaultDS:
    url: jdbc:mysql://localhost:3306/test_database?useUnicode=true&characterEncoding=utf8&serverTimezone=Asia/Shanghai
    username: canal
    password: canal
canalAdapters:
  - instance: prod
    groups:
      - groupId: g1
    outerAdapters:
      - name: rdb
        key: mysql1
    properties:
      jdbc.driverClassName: com.mysql.jdbc.Driver
      jdbc.url: jdbc:mysql://localhost:3306/test_database_01?useUnicode=true&characterEncoding=utf8&serverTimezone=Asia/Shanghai
      jdbc.username: canal
      jdbc.password: canal

Create mapping files under the rdb directory for each table to be synchronized.

dataSourceKey: defaultDS
destination: prod
outerAdapterKey: mysql1
concurrent: true
dbMapping:
  database: test_database_01
  table: test
  targetTable: test_database_01.test
  targetPk:
    id: id
  mapAll: true

4.2 Start canal‑adapter

Enter canal-adapter/bin and start:

cd canal-adapter/bin
./startup.sh & # background

Check adapter/logs/adapter.log to confirm successful startup.

Test Database Synchronization

Perform update, delete, batch insert, batch update, and batch delete operations on the source MySQL database and verify that changes are propagated.

Original source: https://www.jianshu.com/p/d4c177f0d831 (author: qingwenLi)