This article reviews various approaches for keeping MySQL data in sync with Elasticsearch, including direct business‑layer hooks, independent synchronization via plugins or custom scripts, and real‑time binlog subscription using tools like zongji, while discussing their advantages, drawbacks, and implementation details.
This article details the design, architecture, and operational experience of Youzan's Data Platform (DP), covering its scheduling, data‑sync, service, and monitoring modules, the custom Airflow‑based task scheduler, current production metrics, supported task types, and future improvement plans.
This article details Meituan's practical approaches to boosting account service reliability, covering MTBF/MTTR metrics, business‑level monitoring, flexible availability with circuit‑breaker patterns, cross‑region active‑active deployment, data synchronization techniques, and the measurable performance gains achieved.
Meituan/Dianping ensures its critical account service stays online by combining real‑time business monitoring, circuit‑breaker‑driven graceful degradation, and active‑active cross‑region deployment with isolated dependencies, versioned data sync, and automated cache updates, dramatically extending MTBF while cutting MTTR and latency.
This article details the design of a unified MySQL data synchronization platform that consolidates offline sync, real‑time subscription, and real‑time sync into BatchJob, StreamJob, and PieJob abstractions, describing task implementations, cluster architecture, high‑availability mechanisms, and evolution challenges such as file loss and metadata handling.
This article details the evolution of Qunar's TTS pricing engine from a monolithic V1 implementation with dual-level caching and blocking HTTP interfaces to a scalable V2 architecture featuring service separation, asynchronous processing, CQRS-based rule storage, canal synchronization, Solr and Redis indexing, and Kafka-driven data pipelines.
The article describes the requirements, architecture, data‑sync strategy, monitoring, and operational workflow of a search open platform that enables fast, zero‑code creation of searchable interfaces, supporting real‑time indexing, customizable ranking, and extensible backend services.
This article presents a comprehensive technical case study of a large‑scale migration that replaces a Windows/.NET/SQL Server stack with a Linux/Java/MySQL architecture, detailing system decomposition, order service redesign, code conversion scripts, data synchronization challenges, testing strategies, deployment and rollback procedures.
This article explains why an intermediate data structure is needed for 3D rendering in a React‑like Virtual DOM environment, how to keep it reactively synchronized with raw graph data using dependency analysis, proxies, and conversion functions, and the performance benefits of minimal partial updates.
This article presents the design and implementation of a smooth migration solution that moves Baidu's internal bdrp service from Redis 2.6/2.8 to Redis 3.0 by extending the AOF mechanism, enabling real‑time data synchronization, improving service stability, and supporting future upgrades.
This article explores practical techniques for synchronizing heterogeneous data sources, performing batch and incremental analytics with Hadoop and Spark, designing low‑latency real‑time computation pipelines, implementing push notifications, and choosing appropriate storage solutions—from in‑memory caches to distributed databases—while addressing performance, reliability, and scalability challenges.
Taobao’s Open Platform sustains hundreds of billions of daily API calls and messages by employing a pipeline‑based high‑performance API gateway with multi‑level caching, asynchronous processing, granular traffic control, a highly reliable push‑pull messaging system, and a zero‑loss data‑sync service that dynamically balances resources during massive traffic spikes.
Learn how to ensure that a user list page instantly reflects edits made on a separate detail page in a WeChat Mini Program by using a lightweight broadcast utility, avoiding full page reloads and improving user experience.
This article introduces Mongosync, a Qihoo 360‑developed tool for transferring data between MongoDB replica sets, detailing its background, supported features such as full‑data and real‑time oplog synchronization, version compatibility, multithreaded bulk writes, configurable options, performance test results, and future development plans.
This article explains how queues—such as message queues, task queues, and data‑bus queues—are applied in backend systems for asynchronous processing, system decoupling, data synchronization, traffic shaping, buffering, and other architectural patterns to improve scalability and reliability.
A developer discovers a nearly full 1 TB server disk caused by large SQL Server backup files, learns the backup strategy, removes old backups while keeping monthly snapshots, manually syncs production data to a test server, and reflects on backup file formats and storage optimization.
This article explains how to design and implement heterogeneous data synchronization, leverage batch and stream processing frameworks like Hadoop and Storm for large‑scale analysis, and choose appropriate storage solutions—from in‑memory databases to distributed column‑family stores—while addressing performance, reliability, and monitoring in complex distributed environments.
This article details how Tencent's operations team orchestrated a seamless, zero‑impact migration of over 70 million users across three data centers during the 2015 Tianjin explosion, highlighting the four key capabilities—distribution, scheduling, data synchronization, and automated operations—that enabled multi‑active disaster recovery at massive scale.
This guide explains how to enable Alibaba Cloud Data Transmission Service (DTS) with the dts-ads-writer plugin to continuously synchronize changes from an RDS for MySQL instance to a real‑time write table in an analytic database, covering prerequisites, configuration, and monitoring steps.
This article recounts a seven‑year‑old Oracle 10g database migration, detailing project background, team turmoil, topology redesign, security constraints, data‑sync strategies, custom tools, high‑fidelity testing, unexpected failures, and the lessons learned for reliable operations.
This article chronicles how WeChat’s backend evolved from a simple messaging prototype to a globally distributed, multi‑data‑center system, detailing its message model, unified sync protocol, three‑layer architecture, platformization, disaster‑recovery design, performance tuning, and emerging resource‑scheduling challenges.
This article explains why offline capability is essential for mobile apps, outlines key requirements such as error handling and guest mode, and details practical solutions including local caching strategies, queueing requests, data synchronization methods, and recommended libraries.
This article details the evolution of Weibo's database platform—from its early single‑master design through rapid scaling, vertical and horizontal sharding, automation, Redis enhancements, custom middleware, and future plans—offering practical insights into high‑availability, high‑performance data engineering for large‑scale social media services.
This article chronicles the two‑month development of WeChat’s backend from its inception, detailing the design of its message model, data‑sync protocol, three‑tier architecture, asynchronous queues, rapid scaling, platformization, multi‑data‑center deployment, disaster‑recovery strategies, performance optimizations, security hardening, and emerging resource‑scheduling challenges.
The article describes how Alibaba scaled its massive data platform by horizontally sharding Oracle tables to MySQL, employing multi‑data‑center synchronization tools such as Erosa, Eromanga, and Otter, and designing multi‑level caching and proxy solutions like Cobar to ensure high availability and performance.
The article details Alibaba's three‑stage evolution from same‑city active‑active to two‑site three‑center disaster recovery and finally to geo‑distributed multi‑active architecture, explaining its motivations, technical challenges such as latency and data consistency, the solutions implemented, and the resulting scalability and fault‑tolerance benefits.
