Learn step‑by‑step how to set up a binlog‑based MySQL master‑slave replication using Docker‑Compose and Dockerfiles, covering directory structure, configuration files, container creation, verification, and troubleshooting, so you can quickly deploy a reliable MySQL replication environment.
This article explains the MySQL binary log (binlog), its structure and contents, the three main uses—point‑in‑time recovery, master‑slave replication, and audit logging—followed by a detailed walkthrough of the replication process, an overview of Alibaba's Canal mechanism, and extensive code examples illustrating binlog parsing and event handling.
This article explains how Kafka implements transactions, handles under‑replicated partitions, manages high‑watermark and log‑end‑offset evolution, uses leader epochs for consistency, discusses read‑committed isolation, explains why read‑write separation is not supported, and describes delay queues, dead‑letter/retry queues, auditing, tracing, lag calculation, key metrics, and performance‑optimising design features.
This article explains MySQL master‑slave replication, covering its concepts, asynchronous replication mechanisms, various synchronization strategies, step‑by‑step installation and configuration on CentOS 7, and includes common interview questions with answers; it also provides practical commands, troubleshooting tips, and performance considerations for read‑write separation in production environments.
This article explains Kafka's replication process, covering producer acknowledgments, replica synchronization strategies, the roles of ISR and AR, the meanings of HW, LEO, LSO, LW, different ack levels, and how failures are handled to balance reliability and performance.
This guide walks through preparing the environment, configuring the master and slave MySQL servers, and using Python's Paramiko library to automate SSH commands, edit configuration files, restart services, and verify replication, all while handling common pitfalls.
After upgrading a critical MySQL client from 5.6 to 5.7, duplicate‑key errors appeared during inserts on master, replica and read‑only instances, prompting a deep dive into InnoDB auto‑increment mechanics, related kernel bugs, on‑site analysis, and both application‑side and engine‑side remediation strategies.
This article explains how ZooKeeper functions as a distributed coordination service for HBase, detailing its role in master and RegionServer fault tolerance, log splitting, META table location tracking, and replication management, illustrating the underlying ZNode structures and failover mechanisms.
This article walks through preparing a four‑node MySQL environment, configuring password‑less SSH, setting up master‑slave replication with GTID, installing and configuring MHA, testing connectivity, running the manager, verifying automatic failover, and highlighting the key advantages of using MHA for high availability.
This article explains Kafka's internal storage architecture—including topics, partitions, segments, .log and .index files—how data is read, and the various reliability mechanisms such as ISR/OSR, LEO/HW, producer acknowledgment levels, leader election strategies, and delivery semantics.
This article explains MySQL replication (master‑slave and master‑master), data sharding principles and implementations, and various distributed database deployment architectures, covering replication mechanisms, advantages, failure recovery, sharding strategies, middleware like Mycat, and scaling solutions for high‑availability systems.
This article reviews common Kafka interview questions, covering delay queues, idempotence, replica states, offsets, message ordering, and handling duplicate consumption, and includes example code for enabling idempotent producers along with explanations of time‑wheel mechanisms and practical solutions to consumer rebalance issues.
This article describes JDHBase’s large‑scale KV storage, its HBase‑based replication principle, the multi‑active cluster architecture with Fox Manager, client routing, automatic failover, dynamic replication tuning, serial replication guarantees, and future directions for improving cross‑region disaster recovery.
This step‑by‑step guide shows how to use Docker on CentOS 7 to pull MySQL 5.7 images, configure master and slave my.cnf files, run containers, set up replication, and verify that a test database created on the master is automatically synchronized to the slave.
This article provides a comprehensive, step‑by‑step tutorial on creating a Redis Cluster using Docker, covering the preparation of a base Redis image, building node images, configuring cluster settings, assigning hash slots, adding replica nodes for high availability, testing failover, and solving network connectivity issues for production deployments.
This tutorial walks through preparing three CentOS 7 servers, installing and compiling Redis 6.0.3, configuring master‑slave replication, setting up Sentinel for high availability, adjusting firewall and Redis configuration, and managing the services to achieve a fully functional Redis cluster.
This guide outlines step‑by‑step methods to identify high load causes, troubleshoot replication interruptions, resolve instance hangs, and address common MySQL errors using commands like SHOW PROCESSLIST, EXPLAIN, iostat, and GTID management.
This article compares MySQL semi‑synchronous and asynchronous replication, presenting performance test results that show only a small difference in TPS and QPS, explains why the gap is minimal, and provides step‑by‑step instructions and sysbench scripts for setting up and benchmarking both replication modes.
This article explains how to extend Redis beyond single‑node limits by using partitioning, master‑slave replication, Sentinel for automatic failover, and Redis Cluster with hash slots, detailing their usage, advantages, drawbacks, and configuration examples for building high‑availability and scalable in‑memory data stores.
This article provides an in‑depth guide to Redis, covering its core data types, internal data structures, common usage scenarios, differences from Memcached, key expiration, eviction policies, persistence mechanisms, transactions, event handling, replication, Sentinel, sharding, and a practical forum system example.
This article explains Kafka's core architecture, covering how topics and partitions are stored, the advantages of its consumption model, the internal network and threading design, and the high‑reliability distributed log storage and replication mechanisms that ensure data durability and scalability.
This article explains the MySQL group_replication_consistency parameter, describes its five possible values, demonstrates the three most common consistency modes (EVENTUAL, BEFORE, AFTER) with practical multi‑node examples, and discusses their advantages, drawbacks, and impact on query latency and data integrity.
This article explains why MySQL master‑slave replication can suffer from latency, examines the root causes on both master and slave sides, and provides practical architectural, hardware, and configuration solutions—including semi‑sync and parallel replication—to improve data consistency and performance.
This guide explains how to achieve MySQL high availability and scalability through full and incremental backups, binlog-based point‑in‑time recovery, and both asynchronous and synchronous replication, while highlighting trade‑offs, best practices, and practical command examples.
This guide explains MySQL’s fundamentals, describes a master‑slave high‑availability architecture, and provides a complete step‑by‑step tutorial for deploying the cluster on Kubernetes using ConfigMaps, Services, a StatefulSet, dynamic PVCs, and verification commands.
This tutorial walks you through installing MySQL 5.7 on two CentOS 7.6 machines, configuring the master and slave my.cnf files, setting up replication users, starting the services, and verifying synchronization with detailed commands and troubleshooting tips.
Redis 6.0.0 stable release brings major updates such as redesigned client‑side caching, immediate RDB deletion for replication, enhanced ACL logging, faster PSYNC2 partial resynchronization, decimal timeout support, up to 30% faster RDB loading, and a new STRALGO command for string algorithms.
The article analyzes HBase's CP‑oriented CAP limitations, proposes native AP support via Replica, decouples WAL IO to Kafka, optimizes MTTR, introduces multi‑datacenter active/active disaster recovery, and redesigns client write paths and LogSplit processing for higher availability and throughput.
This article provides a comprehensive overview of Hadoop Distributed File System (HDFS), covering its streaming data access model, key characteristics, master‑slave architecture, block storage and replication mechanisms, rack‑aware placement strategy, and how the NameNode manages metadata and checkpoints.
This guide explains the fundamentals of MySQL master‑slave replication, the advantages of using GTID‑based replication, and provides detailed configuration steps, command examples, and troubleshooting tips for setting up and managing a reliable, read‑scalable database environment.
This guide explains how to install Percona XtraBackup 2.4, perform hot full backups of MySQL 5.7 databases, compress and transfer them, and configure master‑slave replication using innobackupex and standard MySQL commands.
This article provides a comprehensive guide to Redis, covering atomic transactions, command pipelining, publish/subscribe messaging, data replication, high‑availability with Sentinel, and the distributed architecture of Redis Cluster, complete with code examples and configuration steps.
This article explains how MySQL computes replication delay using show_slave_status, analyzes the relationship between binlog write timestamps and event generation, enumerates common causes of lag, and provides a detailed examination of the system‑lock state in the slave SQL thread with code examples.
This article explains a MySQL replication experiment where a master is restored from an outdated backup, leading to missing GTID transactions due to server_id filtering, and provides analysis and solutions such as changing server_id or enabling replicate-same-server-id to prevent data loss.
This comprehensive guide covers SQL Server basics such as logging in, creating and dropping databases, tables, indexes and views, advanced query techniques like UNION, outer joins and pagination, plus maintenance tasks, backup and restore, linked‑server data transfer, replication setup, and useful scripts for database administration.
This article provides a comprehensive overview of HDFS, covering its role as a distributed file system, the concepts of blocks, NameNode and DataNode responsibilities, replication, edit logs, snapshots, high‑availability mechanisms, and practical considerations for managing large‑scale data storage.
The article explains the shortcomings of traditional MySQL replication lag measurement, introduces the community heartbeat‑table approach and its limitations, and then details MySQL 8 enhancements (WL#7319, WL#7374) that embed transaction timestamps in binlog events and expose new performance_schema tables, providing precise SQL queries to monitor lag at various stages of the replication pipeline.
This article provides an in‑depth yet approachable overview of Kafka's core concepts—including its architecture, terminology, message‑sending pipeline, replication strategy, controller role, and consumer group rebalance mechanisms—helping readers quickly grasp how Kafka works as a high‑throughput distributed messaging and streaming platform.
This guide walks through preparing the environment, configuring MySQL instances, creating an InnoDB ReplicaSet using MySQL Shell, adding replicas, setting up MySQL Router for read/write splitting, testing connections, and performing common management tasks such as promoting a primary and removing or re‑adding nodes.
This article explains how MySQL's Multi‑Threaded Slave (MTS) processes events, manages checkpoints, and persists state using GAQ queues, bitmaps, and system tables, providing detailed code references and configuration parameters for reliable parallel replication.
This article provides a comprehensive overview of distributed system fundamentals, covering node modeling, replica concepts, consistency levels, data distribution strategies, centralized and decentralized replica protocols, lease mechanisms, quorum, two‑phase commit, MVCC, Paxos, and the CAP theorem, while analyzing their trade‑offs in availability, consistency, and partition tolerance.
JDHBase, JD.com’s large‑scale KV store, powers billions of daily reads and writes across 7,000 nodes, and this article details its multi‑active, cross‑region architecture—including HBase replication fundamentals, Fox Manager routing, automatic failover policies, dynamic replication tuning, and serial replication to ensure strong consistency.
This article explains CouchDB’s eventual consistency model, its use of MVCC, CAP theorem trade‑offs, incremental replication, and document validation, illustrating how these mechanisms enable scalable, high‑availability distributed databases without locking, and includes a practical case study of syncing Songbird playlists.
This article compiles essential Kafka interview material, covering its role as a message queue, usage scenarios, architectural components, storage mechanisms, consumer group rebalancing, high‑availability features, replication details, ordering guarantees, producer/consumer client design, topic management, log retention, performance optimizations, and key monitoring metrics.
MySQL 8.0.19 introduces InnoDB ReplicaSet, Router enhancements, numerous SQL syntax additions, optimizer improvements, new Information Schema role views, security features, replication options, X Protocol compression, and a range of deprecations and bug fixes, providing a comprehensive update for database administrators and developers.
This article explains the principles of MySQL master‑slave replication and provides a detailed, step‑by‑step tutorial for configuring a master and a slave instance in a Docker environment, including configuration files, Docker commands, replication setup, testing, and troubleshooting.
This article dives deep into Kafka’s internal mechanics, explaining how brokers replicate data, how the controller coordinates the cluster via ZooKeeper, the roles of leader and follower replicas, ISR management, request handling, fail‑over strategies, and consumer group rebalancing, all illustrated with diagrams.
This article provides a comprehensive introduction to Redis, covering its key features, supported data structures, core commands, persistence mechanisms, replication, sentinel monitoring, sharding strategies, and transaction handling with practical code examples.
This article describes JDHBase’s large‑scale KV storage deployment, its HBase‑based asynchronous replication mechanism, the multi‑active architecture with active‑standby clusters, client interaction via Fox Manager, automatic failover strategies, dynamic replication tuning, and serial replication techniques to ensure data consistency across data centers.
This guide introduces Redis as a high‑performance key‑value store, covering its core data structures, advantages, persistence mechanisms, publish/subscribe model, transaction handling, and replication strategies with practical command examples.
This article provides a comprehensive technical overview of Kafka's offset handling, covering the request entry point, in‑memory offset sources, offset commit and fetch implementations, file storage layout, and the leader‑follower synchronization process that ensures data replication and high‑watermark updates.
This article explains Kafka's topic architecture, detailing how topics are split into partitions for scalability and parallelism, the role of logs, key-based and round-robin partitioning, replication with leaders, followers, ISR, and how these mechanisms enable fault‑tolerance and high‑performance consumer failover.
This article reviews Alibaba's decade‑long evolution of HBase high‑availability, covering large‑cluster design, MTTF/MTTR metrics, disaster‑recovery strategies, traffic switching, and performance optimizations that together enable millions of requests per second with near‑zero downtime.
This comprehensive guide covers fundamental SQL Server operations such as creating databases, tables, indexes, and views, advanced techniques like set operators, subqueries, and linked servers, plus step‑by‑step instructions for replication, backup, compression, and automated synchronization using stored procedures and SQL Agent jobs.
This article explains the internal calculation of the Time column shown by MySQL's SHOW PROCESSLIST command, covering the source functions, overloads of THD::set_time, various scenarios that affect the result—including replication lag, manual timestamp settings, and idle sessions—and why Percona and official builds may display different values.
This guide introduces a set of scripts for installing MySQL 8.0, configuring master‑slave replication and group replication, detailing required files, command usage, repository location, and essential configuration notes for a three‑node deployment.
This guide explains CloudEndure’s features and walks through a step‑by‑step example of configuring AWS cross‑region disaster recovery, covering initial project setup, data replication, test and recovery mode switching, and the failback process, while highlighting networking and security considerations.
A detailed case study shows how a DELETE statement executed inside a MySQL stored procedure can switch binlog format to ROW, causing billions of bitmap operations on a partitioned table and resulting in severe master‑slave replication delay, and presents three practical solutions.
KeyDB is a high‑performance, multithreaded fork of Redis that retains full Redis compatibility while delivering up to twice the query throughput and 60 % lower latency, and the article explains its thread model, connection management, fastlock mechanism, and active‑replica features in detail.
This article explains how MySQL 8.0.18 introduced privilege checking for replication applier threads, outlines the three-step process to create a restricted‑privilege user, assign necessary permissions or roles, associate the user with the CHANGE MASTER TO statement, and discusses observability and important considerations.
This article explains the structure and purpose of the MySQL slave_relay_log_info table, how relay log positions are persisted and synchronized, the role of master_auto_position and GTID, and the mechanisms of relay_log_recovery to ensure reliable replication after crashes.
This article reviews MySQL replication concepts, compares asynchronous, semi‑synchronous, group, and Percona XtraDB Cluster solutions, explains row‑based versus statement‑based replication, and clears up common misconceptions about replication, clustering, backup, and load balancing.
This article provides a comprehensive MySQL replication FAQ covering cross‑OS and hardware compatibility, master‑slave connection behavior, monitoring lag, forcing master pause, bidirectional replication considerations, performance improvements, high‑availability setups, and how to exclude GRANT/REVOKE statements from replication.
This article provides a comprehensive introduction to Redis, covering its high‑performance key‑value architecture, supported data structures (string, hash, list, set, sorted set), core commands, persistence mechanisms, replication, sentinel monitoring, sharding, pub/sub messaging and transaction handling, with practical command‑line examples.
This article provides a comprehensive overview of HDFS, covering its purpose, architecture, read/write mechanisms, replication strategies, component responsibilities, common command‑line tools, and the advantages and disadvantages of using Hadoop Distributed File System for large‑scale data storage.
This article explains how MySQL's slave_rows_search_algorithms setting and the presence of primary or unique indexes affect the way a replica locates rows for DELETE operations, compares index‑scan, table‑scan, and hash‑scan methods, and provides practical guidance to reduce replication lag.
This guide explains how to configure replication from a MySQL 8.0 master to a MySQL 5.7 slave, covering supported version combinations, character‑set adjustments, authentication plugin changes, user creation, CHANGE MASTER commands, status verification, quick testing, and common pitfalls when using newer MySQL features.
This article provides a comprehensive guide to MySQL performance optimization, covering database design principles, field type selection, indexing strategies, storage engine choices, query caching, partitioning methods, master‑slave replication setup, read/write splitting, load balancing, and profiling tools for diagnosing slow queries.
This article explores Redis's text-based RESP protocol, examines whether Redis truly supports ACID transactions, explains its optimistic lock implementation, details persistence mechanisms like RDB and AOF, discusses memory‑saving data structures, outlines replication processes, and reviews expiration deletion strategies, providing a comprehensive technical overview.
This article provides a comprehensive, step‑by‑step analysis of MySQL’s binlog commit workflow, covering key parameters such as binlog_group_commit_sync_delay, sync_binlog, and binlog_transaction_dependency_tracking, and explains how last_commit and seq_number are generated and used in parallel replication.
The article describes how Tencent Financial Technology (FiT) built a globally largest, financial‑grade distributed MySQL cluster with multi‑active data‑center architecture, Raft‑based failover, priority‑driven switching, and automated backup to achieve 99.999% availability for massive payment workloads.
This article provides a comprehensive overview of Kafka, covering its core features, use‑case scenarios, partition and replica design, file storage structure, consumer‑group coordination, delivery guarantees, performance optimizations, and the role of Zookeeper in managing the cluster.
This article explains Elasticsearch's cluster architecture, including nodes, indices, shards, replicas, deployment models, and data layer storage, and compares two types of distributed data system designs—local file‑system based and shared‑storage based—highlighting their advantages and trade‑offs.
Kafka is a distributed, partitioned, replicated messaging system originally developed by LinkedIn, offering high throughput, low latency, fault tolerance, and scalability; this article explains its core concepts, file storage design, partition replication, leader election, consumer groups, delivery guarantees, and operational considerations for big‑data pipelines.
This article provides a comprehensive overview of Elasticsearch, covering its role in handling structured and unstructured data, core concepts such as Lucene, inverted indexes, clusters, shards, replicas, mapping, indexing processes, storage mechanisms, and practical performance tuning tips for deployment.
This article demonstrates how to install and verify the MySQL 8.0.17 clone plugin, create privileged clone users on both donor and recipient servers, configure cloning parameters, execute a full instance clone, set up replication with CHANGE MASTER, and discusses the plugin's limitations such as InnoDB‑only support and DDL restrictions.
This article explains the architecture of Elasticsearch clusters, covering core concepts such as nodes, indices, shards, replicas, indexing flow, mixed and tiered deployment models, data‑layer storage, and compares these designs with other distributed data system architectures.
This article provides a comprehensive tutorial on MySQL 8.0's Clone Plugin, covering its concepts, local and remote cloning procedures, required permissions, configuration steps, monitoring queries, and a detailed comparison with XtraBackup for fast data replication and recovery.
MySQL 8.0.17 introduces server cloning via MySQL Shell, multi‑value indexes for JSON arrays, enhanced JSON schema validation, optimizer and Volcano iterator upgrades, new collations, replication and routing enhancements, plus numerous bug fixes and deprecations, all detailed in the official release notes.
This article explains how to achieve high‑throughput Redis caching with master‑slave replication, cluster scaling, disk‑less sync, incremental replication, and Sentinel‑based automatic failover to reach 99.99% availability and handle tens of millions of QPS.
This article explains how Redis can be scaled for high concurrency using master‑slave architecture, clustering, and read‑write separation, and it details the mechanisms of replication, persistence, Sentinel monitoring, failover, and configuration that together achieve high availability and data safety.
Redis, the in‑memory data store, uses a simple RESP text protocol, offers limited ACID‑style transactions without rollback, employs optimistic locking via WATCH, provides multiple persistence options (RDB, AOF, hybrid), optimizes memory with ziplist and quicklist structures, and implements sophisticated replication and expiration strategies to balance performance and resource usage.
The article investigates why a DELETE without a WHERE clause on a 500k‑row MySQL table takes over ten hours to replay on a slave, analyzes row‑mode binlog replay mechanisms, compares different slave_rows_search_algorithms, presents test results, and offers practical recommendations for improving replication performance.
This article provides a concise, ten‑minute guide to MySQL master‑slave and master‑master replication, data sharding principles and implementations, and various database deployment architectures—including single‑instance, replication‑based scaling, and sharding‑based scaling—while highlighting practical considerations, advantages, and common pitfalls.
This article explains Kafka's mechanisms for ensuring data reliability through partition replicas, producer acknowledgments, and leader election, and describes how the high‑water‑mark and ISR concepts maintain strong data consistency across the cluster.
The article explains how MongoDB Change Streams, introduced in version 3.6 and expanded in 4.0, enable near‑real‑time subscription to collection, database, and cluster changes, discusses implementation details, options, code examples in the mongo shell and Python, and provides practical testing insights and driver compatibility notes.
The article explains that although Kafka could technically implement a master‑slave (write‑read) model, it deliberately avoids it because the master‑write master‑read design already provides superior load balancing, consistency, and lower latency, making a separate read‑only replica unnecessary.
NebulasFs is a self‑developed distributed, highly available and durable storage system designed to efficiently store billions of small files by using a master‑datanode architecture, multi‑tenant isolation, customizable replication, automatic scaling, and automated replica repair, addressing the challenges of massive unstructured data generated by modern applications.
This article explains the importance of high availability in distributed systems, defines MySQL high‑availability concepts such as replication, describes several HA architectures including MySQL Cluster, MMM, and MHA, and discusses design principles for building resilient database services.
This article explains the MySQL 5.6 slave_rows_search_algorithms parameter, reproduces the HASH_SCAN bug that causes "Can't find record" errors during row‑based replication, analyzes why it occurs, and provides practical solutions such as adjusting keys, changing the algorithm setting, or upgrading MySQL.
This article provides a comprehensive walkthrough of Redis replication, detailing the step‑by‑step sync process, data synchronization mechanisms, full and partial copy workflows, heartbeat design, and asynchronous replication, while highlighting key commands like PSYNC and their practical implications.
This guide explains Redis replication in depth, detailing the step‑by‑step slave‑of initiation, the PSYNC command’s role in data synchronization, the full and partial sync processes, heartbeat mechanisms, and asynchronous write propagation, while highlighting key configuration parameters and performance considerations.
This article provides a comprehensive overview of Kafka's architecture, covering topics, partitions, producer and consumer workflows, replication and leader election, offset management, consumer group coordination, rebalance processes, delivery semantics (at‑most‑once, at‑least‑once, exactly‑once), transactional messaging, and underlying file and configuration details.
This article analyzes common MySQL slave‑lag scenarios—including missing indexes, large master transactions, high write rates, and MyISAM backups—explains their risks, and provides practical solutions such as index tuning, hardware upgrades, configuration changes, and enabling parallel replication to improve replication performance.
This article explains how duplicate MySQL server_id or server_uuid values in replication topologies can cause data loss during high‑availability failover, illustrates the underlying mechanisms with diagrams, and provides practical configuration recommendations to avoid these issues.
The DTLE 2.19.03.0 release introduces cross‑data‑center transfer, cloud synchronization, data aggregation, DDL filtering, schema name conversion, and numerous bug fixes, providing a reliable MySQL‑focused middleware for diverse data replication and migration scenarios.
The article details CynosDB’s cloud‑native PostgreSQL design—using compute‑storage separation, log sinking with asynchronous replay, and multi‑version reads—to enable a one‑primary multi‑read architecture that delivers elastic scaling, reduced I/O, stateless instances, and rapid failover and efficient resource utilization through parallel log‑based recovery.
The article examines MySQL 5.6’s crash‑safe slave replication, explaining how earlier versions’ unsafe relay‑log handling could corrupt position data, describing the atomic update of mysql.slave_relay_log_info via table‑based relay‑log info and transaction coordination, and covering configuration options, recovery behavior, GTID implications, performance trade‑offs, and implementation guidance.
OceanBase, a fully proprietary distributed NewSQL database, has evolved over eight years to support high‑availability, strong consistency via Paxos, flexible replica management, OBProxy routing, LSM‑Tree storage, and migration tools, enabling seamless scaling, disaster recovery, and Oracle compatibility for large‑scale financial services.
The article recounts a pre‑promotion MySQL replication incident at JD.com, explains the master‑slave architecture and thread roles, identifies slow‑SQL and missing indexes as root causes of replication lag, and proposes practical measures to reduce latency and improve system stability.
