This article explains how to use Flink CDC together with the CdcTools utility to automatically capture newly created MySQL tables and synchronize both their schema and data to a Doris database in real time, covering the required code, side‑output handling, async execution, and a special delete‑sign field.
This article explains how a supply‑chain system moved from strict microservice boundaries to a data‑redundancy and real‑time synchronization approach, using Bifrost middleware to keep product information consistent across order and purchase services while improving performance and reducing coupling.
This article analyzes the pain points of inconsistent online‑offline configurations, proposes three synchronization schemes—including timed batch sync, automatic real‑time binlog‑MQ sync, and manual selective sync—compares their applicability, risks and performance, and presents practical deployment results across multiple projects using MySQL and TiDB.
This article explains how Yjs implements operation‑based CRDTs using bidirectional linked lists and a struct store, details the YATA conflict‑resolution algorithm, describes transaction handling, garbage collection, undo management, and the two‑phase network synchronization protocol for building collaborative editing applications.
This article demonstrates how to achieve real‑time synchronization of MySQL data to Redis using Spring Boot 3, the open‑source mysql‑binlog‑connector‑java library, and optional JMX exposure, providing step‑by‑step setup, code examples for parsing binlog events, listening to changes, and configuring Maven dependencies.
This step‑by‑step guide shows how to configure Linux servers for real‑time NFS static resource backup using inotify and rsync, covering host preparation, rsync daemon setup, user and password management, inotify script creation, and verification of synchronized files across multiple machines.
This article details Kuashou's journey of adopting the Hudi data lake, covering business challenges, migration from Hive to Hudi, architectural redesign, promotion strategy, real‑world use cases such as CDC sync and batch‑stream integration, and key lessons learned for large‑scale data engineering.
This article explains a three‑step architecture for describing, storing, and calculating fan‑based discounts, introduces a crowd‑based abstraction to accelerate high‑traffic queries, and details a hybrid offline‑plus‑real‑time data‑sync strategy that ensures consistency and millisecond‑level response times.
This guide explains Rsync's fundamentals, demonstrates local and LAN synchronization via SSH, details daemon deployment and configuration, provides testing commands, and shows how to achieve real‑time bidirectional file sync using inotify‑tools on Linux systems.
DataBus is a real‑time data synchronization framework designed to support Oracle de‑commissioning, micro‑service migration, and heterogeneous storage engines by providing high‑availability CDC, flexible data pipelines, and seamless full‑to‑incremental migration across multiple source and target databases.
This article explains the design of Baidu's transaction accounting platform, covering business scenarios, the flow of transaction records, system architecture, real‑time data synchronization via Canal, Elasticsearch storage strategies, consistency guarantees, and aggregation techniques for accurate merchant financial reconciliation.
The article introduces DingTalk’s Collaborative Engine—a cross‑platform runtime and framework that solves distributed data consistency, enabling real‑time multi‑person collaboration, cross‑device synchronization, and offline‑first capabilities while offering Backend‑as‑a‑Service for developers.
This presentation details Alibaba Cloud DataWorks Data Integration (DataX), covering its architecture, core design principles, offline and real‑time synchronization mechanisms, deployment modes, product positioning, use‑case scenarios, and its role within the broader DataWorks ecosystem, highlighting its capabilities for large‑scale data movement and processing.
This article presents Meituan's comprehensive evaluation of open‑source graph databases, the rationale for selecting NebulaGraph, and the design of a high‑availability, high‑throughput graph database platform that supports multi‑hop queries, massive data ingestion, real‑time synchronization, and visualization for various business scenarios.
Meituan evaluated open‑source distributed graph databases against strict latency, scale, and import criteria, selected NebulaGraph for its superior multi‑hop query and bulk‑load performance, and built a four‑layer, highly available platform that ingests petabyte‑scale data in real time, supports diverse business use cases, and provides interactive visualization.
This guide explains what Canal is, its key features and limitations, the underlying binlog replication principle, and provides detailed, step‑by‑step instructions for downloading, configuring, and launching both Canal Server and Canal Adapter to achieve high‑performance real‑time data synchronization.
This article presents a systematic guide to real‑time MySQL binlog ingestion, outlining three core principles—decoupling from business data, handling schema changes, and ensuring traceability—followed by concrete Debezium‑Kafka‑Hive solutions, scenario‑specific tactics, and practical tips for reliable data pipelines.
This article provides a step‑by‑step tutorial on building a small‑scale data platform by installing MySQL, Zookeeper, Kafka and the open‑source Canal middleware, configuring Canal to capture MySQL binlog events, and forwarding the structured data to Kafka for downstream processing.
This guide walks through configuring two CentOS 7.7 hosts for real-time data synchronization using Sersync and Rsync, covering environment preparation, service deployment, configuration file editing, manual testing, daemon startup, and automatic startup via rc.local.
Facing MySQL performance bottlenecks at massive scale, Xiaomi built the LCSBinlog service that captures MySQL binlog events, streams them through the Talos platform, and writes to Kudu for real‑time BI, detailing architecture, job scheduling, consistency guarantees, use cases, and troubleshooting lessons.
Canal, an open‑source Alibaba project, mimics MySQL slave behavior to subscribe to binlog events, parses them, and supports both standalone and ZooKeeper‑coordinated cluster deployments, offering flexible state storage, message processing pipelines, and integration options such as TCP, Kafka, and RocketMQ for real‑time data synchronization.
The article outlines the challenges of creating multiplayer mini‑games—such as matchmaking, network communication, and stateful deployment—and presents a cloud‑based solution that abstracts room management, online matching, frame‑sync and state‑sync APIs, enabling developers to add real‑time battles to their games within minutes.
This step‑by‑step guide shows how to set up two CentOS 7 virtual machines, install and configure rsync, inotify‑tools, and sersync, and combine them with scripts and cron jobs to obtain reliable, real‑time file synchronization and automated backups across the network.
Aries, an internally built MySQL incremental log distribution platform, leverages GTID‑based binlog dumping to achieve stable, consistent, and real‑time data synchronization across heterogeneous systems, supporting use cases such as Elasticsearch sync, cache updates, archiving, and live statistics.
This article details how Tongcheng Network migrated massive ticketing and flight‑booking workloads to TiDB, combining sharding and a distributed SQL database to handle terabytes of data, real‑time monitoring, and high QPS traffic while simplifying operations and reducing costs.
Transpedia, launched with TDH 4.6, consolidates all StarRing product manuals into a single online portal that offers real‑time synchronization, keyword search, and intuitive navigation, eliminating the need for PDF downloads and dramatically improving documentation accessibility and user experience.
This guide explains how to use Alibaba Cloud Data Transmission and the dts-ads-writer plugin to continuously synchronize MySQL RDS tables to an analytical database, covering prerequisites, configuration files, step‑by‑step setup, and important considerations for reliable data replication.
The article presents an overview of Alibaba's Data Replication Center (DRC), detailing its architecture, real‑time cross‑region synchronization capabilities, consistency and latency guarantees, deployment strategies, and its use cases on Alibaba Cloud such as RDS migration and multi‑active e‑commerce workloads.
Learn how to replace traditional backup methods with rsync and inotify for fast, secure, incremental file synchronization, covering rsync fundamentals, command options, server and client configuration, inotify monitoring scripts, and testing procedures to achieve real‑time data replication between Linux servers.
