Master Data Sync with DBSyncer: From MySQL to Elasticsearch via Docker

Introduction

DBSyncer is an open‑source visual data‑synchronization tool that supports MySQL, Oracle, PostgreSQL, Elasticsearch, Kafka, and file‑based sources.

Key Features

Combination driver: custom source‑to‑target sync, e.g., MySQL → ES.

Real‑time monitoring: full and incremental sync with status, result, and log tracking.

Plugin development: write code to customize transformation logic.

Installation

Use Docker for quick deployment:

docker pull registry.cn-hangzhou.aliyuncs.com/xhtb/dbsyncer:latest

docker run -p 18686:18686 --name=db-syncer \
-e TZ="Asia/Shanghai" \
-v /mydata/db-syncer/data:/app/dbsyncer/data \
-v /mydata/db-syncer/plugins:/app/dbsyncer/plugins \
-d registry.cn-hangzhou.aliyuncs.com/xhtb/dbsyncer:latest

After the container starts, access the web console at http://192.168.3.101:18686 with default credentials admin:admin.

Usage Example – MySQL to MySQL

Create a simplified product table and two databases mall_dev (source) and mall_test (target). Add connections for both databases in DBSyncer, create a driver, map fields (select all), and start the sync.

CREATE TABLE `product` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  `title` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci NULL,
  `sub_title` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci NULL,
  `price` decimal(10,2) NULL,
  `pic` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci NULL,
  PRIMARY KEY (`id`) USING BTREE
) ENGINE=InnoDB AUTO_INCREMENT=2 CHARSET=utf8 COLLATE=utf8_general_ci ROW_FORMAT=Dynamic;

Usage Example – MySQL to Elasticsearch

Create the ES index for the product data:

PUT /product_index
{
  "mappings": {
    "properties": {
      "id": {"type": "long"},
      "title": {"type": "text"},
      "sub_title": {"type": "text"},
      "pic": {"type": "text"},
      "price": {"type": "double"}
    }
  }
}

Add the ES connection in DBSyncer (password optional), create a driver from MySQL to ES, configure field mapping with the primary key, and start the sync. The data becomes searchable in ES (e.g., via Kibana at http://192.168.3.101:5601).

Conclusion

DBSyncer provides a visual way to perform full‑ and incremental‑sync between databases and Elasticsearch, with plugin support for custom logic.

Project Repository

https://gitee.com/ghi/dbsyncer