Tagged articles
data-sync
25 articles · Page 1 of 1
This article explains how to connect Spring Boot to external services by choosing the appropriate HTTP client (RestTemplate, Feign, WebClient), configuring beans, implementing service methods, and applying various data‑synchronization techniques such as full sync, UPSERT, incremental sync, webhook callbacks, and message‑queue based replication.
FeignIncremental SyncMessage Queue
0 likes · 20 min read


ITPUBJan 18, 2026 · Databases
An e‑commerce junior developer tackles the challenge of regularly syncing order data to a data warehouse, evolving from naïve full‑table copies to incremental sync, batch processing, cursor‑based pagination, performance tuning, and finally a real‑time CDC‑plus‑message‑queue architecture, while addressing reliability, ordering, and scaling issues.
CDCCursorbatch
0 likes · 13 min read


java1234Jan 2, 2026 · Backend Development
This guide walks through integrating Spring Boot with external services using RestTemplate, Feign, and WebClient, then compares full, incremental, and real‑time data‑synchronization approaches, providing step‑by‑step code examples, configuration details, and practical trade‑off analysis.
FeignJavaRestTemplate
0 likes · 22 min read

This article presents a comprehensive overview of Flink CDC + Hologres high‑performance data synchronization, detailing write and consumption optimizations, architectural principles, and future directions to achieve low latency and high throughput in real‑time data pipelines.
CDCFlinkHologres
0 likes · 21 min read

This article explores four practical methods for synchronizing MySQL data to Elasticsearch—including synchronous and asynchronous double writes, SQL extraction, and binlog real‑time replication—while reviewing popular migration tools such as Canal, Alibaba DTS, and Databus to help you choose the right solution.
CDCCanalDTS
0 likes · 13 min read

This article explains why MySQL alone struggles with large‑scale, complex queries, introduces Elasticsearch as a high‑performance search engine, and compares several synchronization approaches—including synchronous and asynchronous dual‑write, Logstash, binlog real‑time sync, Canal, and Alibaba Cloud DTS—highlighting their pros, cons, and suitable scenarios.
CanalElasticsearchLogstash
0 likes · 14 min read


ITPUBSep 3, 2024 · Backend Development
A large table with tens of millions of rows caused 20‑second query times, but by decoupling reads through RabbitMQ and writing data asynchronously to Elasticsearch, query latency dropped to 500 ms, with Java code examples for both producer and consumer, sync handling, and migration tips.
ElasticsearchJavaRabbitMQ
0 likes · 6 min read

This article explains the challenges of synchronizing billions of order records to Elasticsearch, outlines the design of the ECP service that automates heterogeneous data integration, and details its implementation including multi‑source reading, SQL parsing, dynamic rate limiting, retry mechanisms, SPI‑based extensibility, environment isolation, health‑check, smooth migration, and logging.
ElasticsearchJavaSPI
0 likes · 19 min read

This article explains how Bilibili's account platform was refactored into a unified multi‑tenant microservice system, covering the background problems, domain‑driven design, data isolation strategies, configuration‑driven tenant onboarding, deployment models, gray‑release migration, and solutions for bidirectional data sync loops.
ConfigurationGray Releaseaccount system
0 likes · 18 min read

This article explains how a government procurement platform built a hybrid cloud‑native “highway” solution using Dubbo to achieve reusable, tunnel‑based data synchronization across cloud and isolated island networks, detailing the background, challenges, design choices, implementation steps, and future roadmap.
Dubbocloud-nativedata-sync
0 likes · 16 min read

This tutorial walks through configuring Canal in cluster mode to capture MySQL row‑based binlog events, publish them to a RocketMQ topic, and consume the ordered messages to update an Elasticsearch index, covering MySQL, RocketMQ, Elasticsearch mappings, and the required Canal properties.
CanalElasticsearchMySQL
0 likes · 8 min read

The article compares four MySQL‑to‑Elasticsearch synchronization approaches—synchronous dual‑write, asynchronous dual‑write via MQ, timer‑based SQL extraction, and binlog‑based real‑time sync—evaluates their trade‑offs, and recommends practical tools such as Canal, DTS, and Databus for implementation.
CanalDTSDatabus
0 likes · 13 min read


ITPUBMay 2, 2023 · Backend Development
This article compares four common approaches for synchronizing MySQL data to Elasticsearch—synchronous dual write, asynchronous dual write via message queues, scheduled tasks, and binlog‑based data subscription—detailing their advantages, drawbacks, implementation steps, and tool choices such as Canal, Maxwell, and Python‑MySQL‑Replication.
CanalElasticsearchMessage Queue
0 likes · 6 min read

This guide explains why traditional mysqldump and file‑based methods fail for massive cross‑database sync, introduces Alibaba’s open‑source DataX middleware, details its framework and plugin architecture, walks through installation on Linux, shows how to configure MySQL source and target, and demonstrates both full and incremental data synchronization with practical JSON job examples.
DataXETLIncremental Sync
0 likes · 14 min read

This guide explores practical strategies for managing Elasticsearch indexes, covering management approaches, structural maintenance, data synchronization, refresh policies, and deep pagination techniques to avoid common pitfalls and ensure reliable search performance.
BackendDeep PaginationElasticsearch
0 likes · 9 min read


ITPUBFeb 12, 2023 · Backend Development
A detailed investigation reveals that RocketMQ can report successful writes to the broker while consumers see minutes‑long delays because the transient store pool and commit‑log forwarding mechanisms delay the transfer of messages from the broker’s memory to the consumer queue.
JavaLag DiagnosisMessage Queue
0 likes · 10 min read

This article details the common pitfalls encountered when synchronizing Oracle databases to AnalyticDB PostgreSQL, covering null versus empty string handling, data type conversion challenges, character set issues, special character processing, and comprehensive testing strategies to ensure data consistency and performance.
AnalyticDBDatabase TestingNULL handling
0 likes · 17 min read

This article examines JD Daojia’s after‑sale platform, detailing its three‑tier architecture, distributed‑lock mechanisms for multi‑endpoint requests, complex promotion‑aware split‑data handling, Elasticsearch synchronization, combined‑order logistics, and rigorous refund validation to ensure accurate and performant service delivery.
BackendDistributed LockElasticsearch
0 likes · 16 min read


ITPUBMar 12, 2021 · Backend Development
This article explains how a growing e‑commerce platform replaced a heavyweight MySQL intermediate table with a binlog‑driven pipeline that streams changes to Elasticsearch via Kafka, detailing the architecture, modules, customizations, monitoring, and performance results.
BackendElasticsearchKafka
0 likes · 13 min read

This guide explains how to use WeChat’s “My Itinerary” feature to view a synchronized map of your travel history, including total distance, trips, and visited cities, by accessing the Train Ticket & Flight service and navigating through the footprint map options.
12306WeChatdata-sync
0 likes · 3 min read

This guide provides a step‑by‑step tutorial on downloading, installing, configuring, and starting Alibaba Canal and its adapter to achieve real‑time incremental data synchronization from MySQL binlog to destinations such as Kafka, including code snippets and configuration details.
CanalDockerKafka
0 likes · 10 min read

This comprehensive guide walks you through setting up a pseudo‑distributed Hadoop environment, loading massive MySQL data with LOAD DATA, Python scripts, and multithreading, and then synchronizing the data to HBase using three approaches—Sqoop, a Kafka‑Thrift pipeline, and a real‑time Kafka‑Flink pipeline—while also comparing query performance of HBase and Phoenix.
FlinkHBaseKafka
0 likes · 28 min read

This article explains why traditional MySQL queries no longer meet the growing e‑commerce data needs, describes the limitations of a MySQL‑to‑Elasticsearch intermediate table, and details a binlog‑driven, Kafka‑based pipeline with custom modules, upsert handling, filtering, and monitoring to ensure fast, reliable data synchronization.
BackendElasticsearchKafka
0 likes · 11 min read

This article details how JD Daojia’s order center migrated from MySQL‑only reads to a multi‑stage Elasticsearch architecture, describing each evolution step, data‑sync strategies, performance pitfalls, and the final real‑time active‑passive cluster that ensures high availability for billions of daily queries.
Cluster ArchitectureElasticsearchHigh Availability
0 likes · 14 min read

This guide walks developers through creating a Wear OS “Hello World” app, handling ambient mode, using BoxInsetLayout, configuring Gradle for phone‑watch communication, implementing MessageApi/DataApi with GoogleApiClient, exploring Wear 2.0 features, and packaging the watch APK for publishing on Google Play.
Android WearGoogleApiClientWear OS
0 likes · 10 min read
