Tagged articles

115 articles

Page 2 of 2

Jul 26, 2020 · Databases

Overview of Canal, Maxwell, Databus, and Alibaba Cloud DTS for MySQL Binlog‑Based Change Data Capture

This article introduces several MySQL binlog-based change data capture solutions—including Canal, Maxwell, Databus, and Alibaba Cloud's Data Transmission Service—explaining their principles, architecture, features, and usage considerations for incremental data subscription and processing.

BinlogCanalChange Data Capture

0 likes · 6 min read

Overview of Canal, Maxwell, Databus, and Alibaba Cloud DTS for MySQL Binlog‑Based Change Data Capture

Big Data Technology & Architecture

Mar 17, 2020 · Big Data

Quick Guide to Building a Canal‑Based Real‑Time Data Synchronization Platform on CentOS 7

This article walks through the end‑to‑end setup of a small‑scale data platform using Alibaba's Canal for MySQL binlog capture, covering the installation and configuration of MySQL, Zookeeper, Kafka, and Canal itself, and demonstrates real‑time change capture with sample DML operations.

Big DataCanalCentOS

0 likes · 20 min read

Quick Guide to Building a Canal‑Based Real‑Time Data Synchronization Platform on CentOS 7

Architecture Digest

Mar 15, 2020 · Big Data

Quick Guide to Deploying Alibaba Canal for Real‑Time MySQL Binlog Synchronization with Kafka and Zookeeper

This article provides a step‑by‑step tutorial on building a small‑scale data platform by installing MySQL, Zookeeper, Kafka and the open‑source Canal middleware, configuring Canal to capture MySQL binlog events, and forwarding the structured data to Kafka for downstream processing.

CanalKafkaZooKeeper

0 likes · 20 min read

Quick Guide to Deploying Alibaba Canal for Real‑Time MySQL Binlog Synchronization with Kafka and Zookeeper

Big Data Technology & Architecture

Feb 16, 2020 · Big Data

Implementing MySQL Binlog Synchronization to HDFS Using Canal

This article details a step‑by‑step guide for deploying Canal to capture MySQL binlog events, configure HA with ZooKeeper, design a client that parses binlog into JSON, asynchronously acknowledges messages, archive data to local files for batch upload to HDFS, and monitor latency for alerts.

Big DataBinlogCanal

0 likes · 10 min read

Implementing MySQL Binlog Synchronization to HDFS Using Canal

Big Data Technology & Architecture

Feb 10, 2020 · Big Data

Real‑time MySQL Binlog Capture with Canal: Principles, Architecture, Deployment and Comparison with Maxwell

This article explains how to use Alibaba's Canal to capture MySQL binlog changes in real time, covering its underlying protocol, component architecture, HA design with ZooKeeper, configuration steps, deployment examples, and a detailed comparison with alternative tools such as Maxwell and mysql_streamer.

Big DataBinlogCanal

0 likes · 17 min read

Real‑time MySQL Binlog Capture with Canal: Principles, Architecture, Deployment and Comparison with Maxwell

Big Data Technology & Architecture

Oct 13, 2019 · Big Data

Installing and Configuring Alibaba Canal for MySQL Binlog Capture

This guide explains how to download, install, and configure Alibaba Canal—including extracting the package, setting up canal.properties, instance.properties, and instance.xml files, and tuning key parameters—to enable reliable MySQL binlog capture for big‑data pipelines.

Big DataBinlogCanal

0 likes · 13 min read

Installing and Configuring Alibaba Canal for MySQL Binlog Capture

Big Data Technology & Architecture

Oct 13, 2019 · Big Data

Building a Simple Canal-to-Kafka Demo with Maven Dependencies and Java Code

This guide introduces the canal‑kafka integration package, outlines its constraints, and provides a step‑by‑step tutorial with Maven dependencies and Java source code for a SimpleCanalClient, a Kafka producer, and a server class, enabling a functional demo of Canal to Kafka data streaming.

Big DataCanalData Integration

0 likes · 8 min read

Building a Simple Canal-to-Kafka Demo with Maven Dependencies and Java Code

Big Data Technology & Architecture

Oct 13, 2019 · Databases

Introduction to Alibaba Canal: MySQL Binlog Parsing and Java Client Implementation

This article introduces Alibaba's Canal project, explaining its background, architecture, and how it parses MySQL binlog for real‑time data subscription, and provides a complete Java client example with code snippets to demonstrate connecting, fetching, and processing binlog events.

BinlogCanalJava

0 likes · 9 min read

Introduction to Alibaba Canal: MySQL Binlog Parsing and Java Client Implementation

Big Data Technology & Architecture

Oct 8, 2019 · Big Data

Real‑time MySQL Binlog Capture and Offline Hive Restoration for Data Warehouse Production

This article describes a complete solution that uses Alibaba's Canal for real‑time MySQL binlog collection, Kafka for transport, and a customized Camus pipeline to load and merge binlog data into Hive, addressing performance, consistency, and delete‑event challenges in large‑scale data warehousing.

BinlogCamusCanal

0 likes · 12 min read

Real‑time MySQL Binlog Capture and Offline Hive Restoration for Data Warehouse Production

Architecture Digest

Sep 24, 2019 · Big Data

Implementation Principles and Architecture of DBus Data Sources (RDBMS and Log Types)

The article explains how DBus ingests data from relational databases and log sources by detailing its extractor, incremental conversion, and full‑pull modules, the use of Canal and Kafka, rule‑based log structuring, the unified UMS message format, and heartbeat monitoring for reliability.

CanalDBusKafka

0 likes · 13 min read

Implementation Principles and Architecture of DBus Data Sources (RDBMS and Log Types)

Ziru Technology

Sep 6, 2019 · Backend Development

How Alibaba Canal Enables Real-Time MySQL Binlog Replication and Incremental Data Sync

Canal, an open‑source Alibaba project, mimics MySQL slave behavior to subscribe to binlog events, parses them, and supports both standalone and ZooKeeper‑coordinated cluster deployments, offering flexible state storage, message processing pipelines, and integration options such as TCP, Kafka, and RocketMQ for real‑time data synchronization.

BinlogCanalJava

0 likes · 11 min read

How Alibaba Canal Enables Real-Time MySQL Binlog Replication and Incremental Data Sync

Youzan Coder

Apr 17, 2019 · Big Data

Order Data Synchronization Architecture at YouZan: From MySQL to ES and HBase

YouZan’s order data synchronization moves changes from MySQL through Canal‑parsed binlogs into a message queue, then uses sequential SeqNo‑based optimistic locking and HBase’s column‑version timestamps to guarantee ordering for both single‑ and multi‑table updates, while a Logstash‑style configurable pipeline feeds ES for search and HBase for detail queries, eliminating ordered‑queue bottlenecks and ensuring high‑throughput consistency.

BinlogCanalDistributed Systems

0 likes · 12 min read

Order Data Synchronization Architecture at YouZan: From MySQL to ES and HBase

Java High-Performance Architecture

Mar 4, 2019 · Databases

Data Heterogeneity Explained: Sharding, Query Dimensions & MySQL Binlog with Canal

This article introduces the concept of data heterogeneity in large-scale systems, explains how sharding creates query challenges, describes query‑dimension and aggregation heterogeneity, and details implementation techniques such as subscribing to MySQL binlog and using Alibaba’s Canal for reliable data synchronization and storage.

BinlogCanaldata heterogeneity

0 likes · 7 min read

Data Heterogeneity Explained: Sharding, Query Dimensions & MySQL Binlog with Canal

Meituan Technology Team

Dec 6, 2018 · Big Data

Real-time Binlog Collection and Offline MySQL Data Restoration for Data Warehousing

The article presents a CDC solution that combines Alibaba’s Canal for real‑time MySQL binlog capture into Kafka with LinkedIn’s Camus for hourly Kafka‑to‑Hive loading, then merges snapshots and incremental binlog data to accurately and efficiently rebuild ODS tables, supporting sharding and delete events.

BinlogCDCCamus

0 likes · 14 min read

Real-time Binlog Collection and Offline MySQL Data Restoration for Data Warehousing

dbaplus Community

Dec 18, 2016 · Big Data

How DWS Uses Log‑Based Architecture for Real‑Time Data Integration

This article explains the design and implementation of the DWS platform, detailing its log‑driven architecture with Dbus, Wormhole, and Swifts, the technical choices behind real‑time data extraction, transformation, and delivery, and real‑world use cases in finance.

CDCCanalKafka

0 likes · 22 min read

How DWS Uses Log‑Based Architecture for Real‑Time Data Integration