Tagged articles
Flink CDC
35 articles · Page 1 of 1

ITPUBSep 14, 2026 · Big Data
This article compares Flink CDC with traditional Canal-based architectures, explains Flink CDC's core principles including lock-free snapshots and exactly-once semantics, provides DataStream and SQL code examples for MySQL integration, and covers five high-frequency interview questions on DDL handling, DELETE capture, and large-table optimization.
DebeziumFlink CDCFlink SQL
0 likes · 12 min read

This article details migrating from a legacy Canal+Kafka+Flink+Hive stack to a modern Flink CDC + Iceberg real-time data lake, covering Spring Boot orchestration, chunk-based full/incremental sync, Interval Join for wide tables, automated schema evolution, hidden partitioning, Exactly-Once checkpoint tuning, and Kubernetes deployment practices.
Apache IcebergFlink CDCKubernetes
0 likes · 17 min read

This guide walks through setting up a Spring Boot 3.5 environment, configuring Flink 1.20 and Flink CDC 3.5, preparing MySQL tables, and using both the Flink CDC CLI and SQL client to achieve near‑millisecond synchronization of data from MySQL to Elasticsearch, including custom sink programming and real‑time monitoring via the Flink Web UI.
Apache FlinkElasticsearchFlink CDC
0 likes · 14 min read

The article provides a comprehensive overview of Change Data Capture (CDC), detailing its purpose, underlying mechanisms across major databases, popular CDC tools, typical use‑cases such as real‑time sync and cache updates, and a reference architecture for building reliable CDC pipelines.
CDCDebeziumFlink CDC
0 likes · 12 min read

This article analyzes the architecture, sync modes, latency, scalability, usability, and deployment aspects of four popular data synchronization solutions—Sqoop, DataX, Flink CDC, and Airbyte—and provides a practical decision tree to avoid common misuse pitfalls in enterprise data pipelines.
AirbyteDataXFlink CDC
0 likes · 9 min read


ITPUBJan 29, 2026 · Big Data
This guide explains how to extend a Flink CDC pipeline so that, in addition to real‑time data replication, DDL ALTER statements from MySQL are captured, split from the data stream, and applied to Doris using side‑outputs and a custom JDBC sink.
DDL synchronizationFlink CDC
0 likes · 8 min read

This article details a full-chain optimization of a Flink CDC real-time data warehouse pipeline syncing 20 MySQL instances to StarRocks via Kafka, resolving GC crashes, checkpoint failures, and 30-minute latency through resource scaling, parallelism redesign, memory tuning, checkpoint configuration, and startup strategy changes, achieving 5-second end-to-end latency and 100% checkpoint success.
Checkpoint TuningFlink CDCGC optimization
0 likes · 18 min read

This article details a production Flink CDC server_id conflict caused by copied configurations, explaining how to identify duplicate server_id/server_uuid via MySQL SHOW PROCESSLIST, assign unique identifiers, clear Flink checkpoint state, and verify resolution through logs and replication monitoring.
DebeziumFlink CDCMySQL
0 likes · 9 min read

This article presents Huolala’s end‑to‑end implementation of Flink CDC for real‑time data capture, detailing the business background, reasons for selecting Flink CDC over Canal, component comparisons, production‑level platform enhancements, data‑lake integration, validation methods, and future directions for unified data ingestion.
Flink CDCdata synchronizationreal-time-data
0 likes · 13 min read

This article details the background, configuration, and step‑by‑step troubleshooting process for a data pipeline that replaces a MySQL source with OceanBase using OBLogProxy in binlog mode, explains why downstream reads missed incremental data, and provides conclusions and optimization measures.
Flink CDCOBLogProxyOceanBase
0 likes · 12 min read

This article presents a comprehensive overview of Alibaba Cloud's modern data stack built on Flink CDC, detailing its core concepts, extended capabilities, typical application scenarios, performance optimizations, a live demo, and future development plans for large‑scale streaming data integration.
Alibaba CloudFlink CDCbig data
0 likes · 13 min read

Alibaba Cloud DataWorks’ Data Integration platform, built on Flink CDC, offers a comprehensive, serverless solution for real‑time and batch data lake ingestion, detailing its architecture, elastic scaling, productized use cases, and future roadmap, including AI‑driven diagnostics and expanded source support.
Elastic ScalingFlink CDCbig data
0 likes · 12 min read

This article details HuoLaLa's logistics platform challenges with petabyte‑scale data, the selection of Apache Flink CDC for stable, compatible, and low‑latency data ingestion, the construction of a multi‑layer CDC capability, migration strategies, measurable performance gains, and future open‑source contributions.
Apache FlinkData IngestionFlink CDC
0 likes · 15 min read

The article explains how to prepare for interview "if" questions about Binlog loss by describing Flink CDC's binlog extraction principles, possible recovery mechanisms, and practical strategies such as resetting offsets, extending log retention, and building offline‑online reconciliation pipelines.
Data StreamingFlink CDCInterview
0 likes · 4 min read

The article explains how Flink CDC 3.0 transforms traditional CDC pipelines into an end‑to‑end streaming ELT framework, offers practical guidance for describing such projects on resumes and in interviews, and outlines future challenges and development directions for large‑scale data integration.
Flink CDCbig datadata integration
0 likes · 6 min read

The Flink Forward Asia 2023 conference showcased major updates to Apache Flink (versions 1.17 and 1.18), introduced the Apache Paimon lakehouse project, announced Flink CDC 3.0, and highlighted community growth, cloud‑native deployments, and real‑time data‑warehouse use cases across industry leaders.
Apache FlinkApache PaimonFlink CDC
0 likes · 17 min read

The Flink Forward Asia 2023 conference recap highlights opening remarks, a keynote on Flink’s dominance in streaming compute, detailed 2023 technical advancements, case studies, the launch of Flink CDC 3.0, and a preview of Flink 2.0, along with links to photos and video recordings.
Apache FlinkFlink 2.0Flink CDC
0 likes · 5 min read

This article, based on a presentation by Flink CDC and Apache Flink community leaders, explores CDC real‑time integration challenges, delves into Flink CDC’s core technologies such as incremental snapshot and lock‑free processing, and demonstrates Alibaba Cloud’s enterprise‑grade solutions for end‑to‑end real‑time data pipelines.
Alibaba CloudFlink CDCReal-Time Data Integration
0 likes · 21 min read

This article explains how to build a no‑code, configurable real‑time data synchronization system using Flink CDC, covering feasibility, core features, UI workflow, task submission, and practical considerations.
Apache DorisFlink CDCKafka buffering
0 likes · 9 min read

This article introduces LakeSoul, an open‑source end‑to‑end real‑time lakehouse framework, detailing its design philosophy, key technologies such as ELT, metadata management, upsert and merge‑on‑read capabilities, performance benchmarks, real‑world use cases, and the roadmap for future enhancements.
ELTFlink CDCLakeSoul
0 likes · 18 min read

This article explains data integration fundamentals, compares data integration tools such as Stitch, Fivetran, and Airbyte, describes the concepts of data warehouses and data lakes, outlines ETL vs ELT processes, and explores building modern data stacks with Flink CDC and cloud services.
ELTETLFlink CDC
0 likes · 17 min read

This article introduces Flink CDC, explains its incremental snapshot algorithm and the 2.0 framework design, compares it with traditional CDC pipelines, discusses the core API and dialect concept, and outlines community growth and future plans, providing a comprehensive technical overview for data engineers.
Apache FlinkData StreamingFlink CDC
0 likes · 13 min read

This article explains how Alibaba Cloud EMR‑StarRocks integrates with Flink CDC, outlines common real‑time ingestion pain points, and introduces the CTAS/CDAS and Connector‑V2 features that streamline table creation, schema evolution, and resource‑efficient streaming for large‑scale analytics.
CDASCTASConnector V2
0 likes · 14 min read

This article, based on a Flink CDC Meetup talk by OceanBase expert Wang He, introduces OceanBase’s architecture and core features, explains the implementation principles of the Flink CDC OceanBase connector, outlines its current capabilities and use cases, and discusses future enhancements and roadmap.
ConnectorDistributed DatabaseFlink CDC
0 likes · 15 min read

This article explains how to construct a real‑time data warehouse by combining Flink CDC for end‑to‑end change data capture with StarRocks' high‑performance OLAP engine, detailing the architectural challenges, optimization techniques, and a practical e‑commerce case study.
Flink CDCOLAPStarRocks
0 likes · 16 min read

This article details how a fast‑growing supply‑chain platform migrated from MySQL and Hive to Apache Doris for real‑time analytics, describing the architectural evolution, the advantages of the new design, practical implementation steps, encountered challenges, and the performance and cost benefits achieved.
Apache DorisFlink CDCMySQL
0 likes · 12 min read

This article explores how combining Flink CDC with StarRocks can streamline real‑time data pipelines, reduce component complexity, support both full and incremental synchronization, and enable efficient OLAP queries and updates for fast, scalable analytics across diverse business scenarios.
Flink CDCOLAPStarRocks
0 likes · 18 min read

Facing server‑hardware constraints, Bilibili’s data platform replaced wasteful full‑partition tables with a zipper‑table approach—preserving change history while cutting storage from petabytes to terabytes—and complemented it with Hudi + Flink CDC for near‑real‑time updates, dramatically lowering I/O, compute usage and latency.
Flink CDCHudiPartition Table
0 likes · 11 min read

The article introduces Flink CDC 2.2, highlighting its expanded support for twelve data sources—including OceanBase, PolarDB‑X, SqlServer, and TiDB—while detailing core features such as the incremental snapshot framework, multi‑version Flink compatibility, dynamic table addition, and numerous bug fixes and performance improvements.
Apache FlinkConnectorFlink CDC
0 likes · 9 min read

This article introduces the fundamentals of Flink CDC, explains its application scenarios and underlying technologies, compares query‑based and log‑based CDC, showcases open‑source solutions, and provides detailed Java and SQL examples for building real‑time ETL pipelines with MySQL and Flink.
Apache FlinkETLFlink CDC
0 likes · 24 min read

In an exclusive InfoQ interview, Apache Flink community leader Wang Feng (aka Mo Wen) outlines the evolution of Flink toward a Streaming Warehouse, detailing recent technical advances, use‑case scenarios, and the upcoming Dynamic Table storage that aim to unify stream and batch processing for real‑time data‑warehouse workloads.
Apache FlinkDynamic TableFlink CDC
0 likes · 16 min read

This interview explores Apache Flink’s evolution toward a Streaming Warehouse, detailing its stream‑batch integration, new CDC‑based data integration, the Dynamic Table storage architecture, and how these innovations aim to simplify and accelerate real‑time big‑data analytics.
Apache FlinkDynamic TableFlink CDC
0 likes · 17 min read

This article explains how to integrate Apache Hudi with Flink CDC to build a near‑real‑time data lake, covering Hudi’s storage model, streaming primitives, version compatibility, Maven setup, SQL table definitions, data flow from MySQL through Kafka, and practical troubleshooting tips.
Apache HudiFlink CDCReal-time Data Lake
0 likes · 18 min read

This article provides a detailed comparison of three popular open‑source change data capture tools—Debezium, Flink CDC, and Canal—covering their underlying principles, architecture, deployment options, performance characteristics, and suitability for real‑time data synchronization in big‑data environments.
CDCCanalDebezium
0 likes · 21 min read

This article provides an in‑depth technical overview of Flink CDC 2.0, covering its CDC fundamentals, comparison of query‑based and log‑based approaches, the new lock‑free chunk algorithm, FLIP‑27 based parallel snapshot reading, performance benchmarks, documentation improvements, and future roadmap for stability and ecosystem integration.
DebeziumFlink CDCMySQL
0 likes · 16 min read
