Understanding Data Heterogeneity: Scenarios, Methods, and Implementation with Binlog, Canal, and MQ

This article explains the concept of data heterogeneity, outlines common use cases such as sharding and multi‑dimensional queries, and details practical implementation methods including full cloning, marked sync, binlog‑based replication with Canal, and MQ‑driven approaches, while providing deployment tips and references.

CanalElasticsearchMessage Queue

0 likes · 12 min read

Understanding Data Heterogeneity: Scenarios, Methods, and Implementation with Binlog, Canal, and MQ

Dada Group Technology

May 21, 2021 · Backend Development

Implementing Data Heterogeneity for JD Daojia Order Fulfillment: Architecture, Canal Integration, and Lessons Learned

This article examines JD Daojia's order fulfillment system, detailing the challenges of high‑volume prompt‑sound queries, the division of responsibilities among Redis, MySQL, and Elasticsearch, the adoption of Canal for asynchronous data replication, deployment practices with Kafka and Zookeeper, and the key operational lessons learned.

Backend ArchitectureCanalElasticsearch

0 likes · 14 min read

Implementing Data Heterogeneity for JD Daojia Order Fulfillment: Architecture, Canal Integration, and Lessons Learned

dbaplus Community

May 10, 2021 · Backend Development

How JD.com’s Order Fulfillment Scales: Data Heterogeneity & Complexity‑Driven Architecture

This talk explains JD.com’s order‑fulfillment workflow, the underlying storage stack of Redis, MySQL and Elasticsearch, the challenges of high‑traffic alert‑sound queries, the adoption of Canal for data heterogeneity, and a 4R‑based complexity‑oriented design methodology for building resilient backend systems.

Backend ArchitectureCanalElasticsearch

0 likes · 18 min read

How JD.com’s Order Fulfillment Scales: Data Heterogeneity & Complexity‑Driven Architecture

JD Retail Technology

Apr 18, 2019 · Big Data

Data Heterogeneity with BinLake, Binlog, and Flink: Approaches for Order, Subscription, and Product Data

The article explains how data heterogeneity is achieved using JD's BinLake to capture MySQL binlogs, with Flink handling sequential and parallel consumption for order, subscription, and product data, discussing challenges such as ordering guarantees, idempotency, IO overhead, and the shift toward stream‑processing architectures.

BinlogElasticsearchFlink

0 likes · 5 min read

Data Heterogeneity with BinLake, Binlog, and Flink: Approaches for Order, Subscription, and Product Data

Java High-Performance Architecture

Mar 4, 2019 · Databases

Data Heterogeneity Explained: Sharding, Query Dimensions & MySQL Binlog with Canal

This article introduces the concept of data heterogeneity in large-scale systems, explains how sharding creates query challenges, describes query‑dimension and aggregation heterogeneity, and details implementation techniques such as subscribing to MySQL binlog and using Alibaba’s Canal for reliable data synchronization and storage.

BinlogCanaldata heterogeneity

0 likes · 7 min read

Data Heterogeneity Explained: Sharding, Query Dimensions & MySQL Binlog with Canal