This article introduces distributed locks, explains why they are needed in distributed environments, outlines their characteristics and requirements, and compares three implementation approaches—plain Redis, ZooKeeper, and Redisson—detailing their advantages, drawbacks, and code examples.
This article provides a comprehensive introduction to ZooKeeper, covering its definition, ZNode hierarchy, listener mechanisms, and practical applications such as unified configuration management, naming services, distributed locks, and cluster state monitoring, illustrating how these features enable robust coordination in distributed systems.
This article explains why traditional in‑process locks fail under massive concurrency, how distributed locks using Redis or Zookeeper ensure atomic updates across many servers, and details practical implementations, pitfalls like deadlocks, and strategies such as key expiration and ephemeral sequential nodes.
Distributed locks provide mutual exclusion across multiple nodes in scalable, reliable systems, and can be implemented via simple Redis SETNX/EX patterns, ZooKeeper’s EPHEMERAL_SEQUENTIAL znodes with watches, or Redisson’s advanced Java‑centric primitives, each offering distinct trade‑offs in simplicity, consistency, and feature richness.
Microservices replace monolithic apps with independently deployable services, but introduce complexity that NetEase Media addresses by unifying frameworks—choosing Consul for service discovery, combining in‑process and sidecar invocation, employing Hystrix for fault tolerance, and using Kong as a unified API gateway to streamline migration across Java, C++, and Python stacks.
This article explains the ZAB protocol—Zookeeper's atomic broadcast and crash‑recovery mechanism—detailing its design, message‑broadcast process, leader election, transaction ordering with ZXID, and how it ensures data consistency and availability in distributed systems.
This article explains how Zookeeper can be used to build distributed locks, detailing the implementation of exclusive locks and two variants of read‑write locks, complete with step‑by‑step processes, node structures, watcher mechanisms, and practical considerations for backend systems.
Zookeeper, originally a side‑project of Hadoop, is a Yahoo‑developed distributed coordination framework that provides high‑availability services such as configuration management, distributed locks, and failure handling, and has become a foundational component for many big‑data systems like Hadoop, Kafka, and Dubbo.
This guide explains how to use the nginx‑upstream‑reloader module together with Zookeeper and the dyups module to automatically discover backend services, persist upstream configurations, and update Nginx without reloads, even when IP addresses change frequently.
This article details UCloud's five‑month, multi‑department hot migration of its Seoul data center, covering planning, ZooKeeper scaling, udatabase and MySQL migration strategies, deployment platforms, and the final cut‑over steps that ensured zero user impact.
The article explains how the CAP theorem forces trade‑offs between consistency, availability and partition tolerance, then surveys distributed commit protocols (2PC, 3PC) and consensus algorithms (Paxos, Raft, Zookeeper’s ZAB), and shows their practical use in systems such as ZanKV that combine Raft with RocksDB for strongly consistent, fault‑tolerant key‑value storage.
This article provides an in‑depth technical overview of Apache Storm, covering its core architecture, key components such as Nimbus, Supervisor, Worker, Executor, and Task, the role of ZooKeeper, high‑availability setup, API interfaces, code examples, grouping strategies, metrics, back‑pressure handling, and essential configuration parameters for building low‑latency stream processing topologies.
This article introduces CIM, a cross‑platform instant messaging framework, detailing its architecture, core components, registration and routing mechanisms, client‑server interaction, custom protocol, built‑in commands, and step‑by‑step deployment using SpringBoot, Netty, Redis and Zookeeper.
This article explains the origins of service registration and discovery in microservice architectures, outlines the problems they address, describes their key characteristics, and compares implementation approaches such as DNS, Zookeeper, Dubbo, and Eureka, highlighting the CAP‑theorem trade‑offs between Zookeeper (CP) and Eureka (AP).
This article provides a comprehensive analysis of Pigeon, a point-based open-source RPC framework used by the company, covering its client-side invoker, server-side provider, call flows, Zookeeper integration, service isolation, and operational features.
This article compares Google’s Chubby lock service with Apache Zookeeper, explains their design philosophies, system architectures, the Zab protocol versus Paxos, and demonstrates how Zookeeper can be used for naming services, publish‑subscribe, distributed transactions, and distributed locks with concrete Java code examples.
This article compares various distributed unique identifier generation methods—including UUID, database auto‑increment, Redis INCR, Zookeeper, segmented DB caches, and the Snowflake algorithm—detailing their principles, advantages, drawbacks, and suitable use‑cases to help developers select the optimal solution for their systems.
This article explains the principles, requirements, and implementation details of distributed lock services, comparing Redis and ZooKeeper approaches, and introduces SharkLock's design built on SharkStore with Raft-based replication, covering lock acquisition, release, reliability, scaling, and failover mechanisms.
This article explains the principles of distributed locks, compares Redis‑based and ZooKeeper‑based implementations, discusses their limitations, and introduces the SharkLock system built on SharkStore with Raft‑based replication, detailing its reliability, dead‑lock prevention, and safety mechanisms.
This article provides a comprehensive overview of ZooKeeper, covering its purpose as a high‑availability coordination service, common application scenarios, detailed architecture roles, node types, session and watcher mechanisms, core characteristics, workflow, and essential query commands.
The article explains that Eureka 2.0's open‑source development has been halted, advises developers to assess whether they are using the discontinued 2.x version, and outlines safe migration paths to alternatives like Consul, ZooKeeper, or Etcd while emphasizing cautious decision‑making.
This article explains the Dubbo consumer initialization workflow, detailing how ReferenceConfig creates proxies, how service URLs are registered with Zookeeper, the steps to obtain Invoker objects via Protocol, and how ProxyFactory generates client-side service proxies, including code examples and asynchronous handling.
This article explains the concept of distributed locks and provides step‑by‑step implementations using Redis's SET command with NX/EX options and Zookeeper's persistent and sequential znodes, including lock acquisition, release, and failure handling.
This article explains Zookeeper's leader election mechanism, covering server identifiers, election states, vote structures, the two types of leader elections, the FastLeaderElection algorithm, quorum calculation, and the underlying network and queue components that enable reliable distributed consensus.
This article explains the concept of sequential consistency, its origins in Lamport's work, how it differs from linearizability, and why Zookeeper adopts sequential consistency for its coordination services, illustrating the theory with concrete examples and code snippets.
This article introduces Dubbo’s background, explains the evolution of e‑commerce architectures to RPC‑based distributed systems, details Dubbo’s components, advantages, and drawbacks, and provides a complete Maven‑based demo—including Zookeeper installation, Spring configuration, and Java code—for building and consuming a Dubbo service.
This article compares three popular service discovery solutions—Zookeeper, etcd, and Consul—detailing their architectures, features, integration methods, strengths, and weaknesses, and concludes with a recommendation for using Consul in multi‑data‑center environments while noting its real‑time notification limitations.
A staging environment’s new scheduled task kept failing due to Zookeeper disconnections caused by packets exceeding the default 1 MB maxBuffer, and the article explains the root cause, heartbeat timing, and how adjusting Djute.maxbuffer or upgrading Zookeeper resolves the issue.
This article explains why distributed locks are needed in Java applications, outlines the essential conditions they must satisfy, and compares three practical implementations—database‑based, Redis‑based, and Zookeeper‑based—providing code snippets and performance considerations for each.
This article explains how to replace traditional in‑process locks with a Redis‑backed distributed lock service, detailing the overall architecture, server and client designs, management features, lock lifecycle, grading strategies, and fallback options such as database and Zookeeper solutions.
This article explains ZooKeeper's distributed coordination architecture, the ZAB consensus protocol, server roles, write and read workflows, FastLeaderElection mechanics, configurable election algorithms, and how ZooKeeper can be used to implement reliable distributed locks and leader election.
This article explores how distributed systems determine node liveness, manage failover and recovery, and implement at‑most‑once, at‑least‑once, and exactly‑once processing guarantees—including opaque transactions and two‑phase commit—using examples from Kafka, Zookeeper, and big‑data pipelines.
This article explains why Kafka needs high availability, how data replication and leader election work together to keep partitions serving despite broker failures, and details the Zookeeper structures and broker failover process that enable Kafka's robust HA design.
This guide walks through installing prerequisites, configuring Zookeeper as a registry, setting up Dubbo, and deploying Tomcat to access the Dubbo web console on a CentOS 6.9 environment.
This article presents a multiplexed system testing environment that uses token‑based encoding and decoding to share a single baseline environment among multiple testing scenarios, detailing its architecture, technical components, protocol adaptation, token handling, and Zookeeper‑based configuration management to improve resource efficiency and testing speed.
This guide walks you through installing Zookeeper and Kafka on a CentOS server, configuring essential settings, creating topics, and running producers and consumers, while highlighting common pitfalls and providing the exact commands needed for a successful deployment.
This step‑by‑step guide shows how to install and configure Apache Kafka and Zookeeper on Ubuntu Server, covering single‑node setup, a pseudo‑cluster with multiple brokers, and a full distributed cluster across several machines, including topic creation, producer/consumer testing, and essential configuration tweaks.
This article introduces Apache Kafka as a fast, scalable distributed publish‑subscribe system, explains its core components, Zookeeper coordination, startup workflow, key features, and common scenarios such as log collection, activity tracking, and stream processing.
This article examines the evolution of distributed locking—from simple MySQL table locks to Redis cache locks and the Redlock algorithm—highlighting their limitations, expert criticisms, and why Zookeeper often provides a more reliable solution for high‑availability systems.
This article provides a comprehensive, step-by-step tutorial for setting up a high‑availability Hadoop cluster, covering user creation, JDK installation, host configuration, SSH setup, firewall and SELinux adjustments, Zookeeper deployment, HDFS and YARN HA configuration, essential XML files, and failover testing.
This article explains how to replace Redis‑based lock with Curator's InterProcessMutex, detailing its re‑entrant design, node creation, lock acquisition logic, waiting mechanisms, and release process, while highlighting advantages over traditional Thread.sleep approaches in multithreaded resource access.
This article demonstrates how to use Zookeeper as a distributed stepwise queue to coordinate multiple Hadoop MapReduce jobs for purchase, sales, and other cost calculations, automatically triggering a profit computation once all tasks complete, and provides full Java code examples and deployment instructions.
This article explains how ZooKeeper’s ServerCnxn, particularly the NIOServerCnxn implementation, handles client connections, processes session creation, manages sessions via SessionTracker, and routes transaction requests through the server’s request‑processor chain, detailing the roles of each processor and the underlying logging and snapshot mechanisms.
This article explains Kafka’s core architecture—including brokers, topics, partitions, offsets, producer and consumer mechanics, replication, availability, consistency, persistence, performance optimizations, and Zookeeper integration—providing a comprehensive guide for building reliable distributed messaging systems.
This guide details the step‑by‑step process for replacing a faulty ZooKeeper node (myid 5) in a five‑node cluster, covering configuration updates in zoo.cfg, Hadoop’s hdfs‑site.xml, yarn‑site.xml, HBase‑site.xml, and the required service restarts to ensure continuous high‑availability.
This article describes the challenges and solutions behind designing a high‑availability, low‑latency ranking service that supports tens of thousands of leaderboards, optimizes storage engine choices, automates scheduling, and isolates resources using ZooKeeper, Redis, and container‑based deployments across multiple data centers.
This article uses a vivid island metaphor to break down the Paxos consensus algorithm, maps its concepts to Zookeeper components such as servers, leaders, and Zxid, and illustrates conflict resolution and leader election through clear examples and diagrams.
This article explains the design and implementation of XDB, a MySQL high‑availability framework used by Baidu Waimai, detailing its component architecture, Horus state‑machine based failover, monitoring mechanisms, log management, and real‑world deployment experiences.
This article explains microservice architecture versus monolithic design, discusses communication, service discovery, ordering, and messaging strategies, and details GeTui’s practical implementation using OpenResty, API gateways, Zookeeper, and DevOps automation.
This article explores the challenges of mutual exclusion and idempotency in distributed environments, explains the underlying principles of locks in multi‑threaded and multi‑process contexts, and presents practical implementations using Zookeeper, Redis, Tair, and the Cerberus and GTIS frameworks to ensure reliable, scalable operations.
This article revisits the Redlock debate by examining Antirez's rebuttal to Martin Kleppmann, analyzing Hacker News discussions, comparing Redis, ZooKeeper, and Chubby lock implementations, and exploring timing assumptions, clock skew, and fencing token mechanisms to assess distributed lock safety.
This article explains how to standardize and automate MySQL management at scale, covering dedicated instance deployment, configuration consistency, multi‑instance creation, metadata collection, backup, monitoring, high‑availability with Zookeeper, and task orchestration using DBTask to achieve rapid, reliable database services.
This article provides a comprehensive guide to Zookeeper, covering its purpose in distributed systems, step‑by‑step installation for both standalone and cluster modes, detailed configuration options, core data model, essential Java APIs, and typical scenarios such as naming service, configuration management, leader election, locks, and queue handling.
This article reviews the evolution of distributed locking—from simple MySQL table locks to Redis cache locks and the Redlock algorithm—examines expert criticisms of Redlock’s correctness, presents the Redis author’s rebuttal, and ultimately recommends Zookeeper as a more reliable solution for high‑availability distributed locks.
YARN’s ResourceManager is a single point of failure, so this guide explains how to configure active/standby mode using Zookeeper, covering leader election, automatic failover, handling false‑dead scenarios, and the essential Zookeeper features such as temporary nodes, watches, and ACLs.
This tutorial explains why Dubbo was created for large‑scale distributed services, how it solves URL management, load‑balancing and capacity‑planning challenges, and provides step‑by‑step instructions for integrating Dubbo with Zookeeper, dubbo‑admin, and Spring MVC using Maven and Java code.
The article examines how distributed systems face mutual‑exclusion and idempotency challenges, explains traditional thread and process synchronization, then details distributed‑lock techniques (e.g., Zookeeper, Redis, Tair, Cerberus) and global‑ID‑based idempotent services, emphasizing the importance of external storage, fault‑tolerance, and proper lock granularity for reliable high‑throughput applications.
This talk explains how Lianjia redesigned its MySQL high‑availability setup by replacing VIP‑based failover with a Zookeeper‑driven naming service, detailing the original MHA architecture, its shortcomings, the new components, workflow, operational challenges, and a DNS‑based refinement.
Pinterest shares its real‑world experience using ZooKeeper for service discovery and dynamic configuration, detailing the challenges of connection overload, transaction spikes, network partitions, and human error, and explains the multi‑step solutions that improved resilience and reduced operational risk.
ZooKeeper's leader election mechanism creates temporary sequential nodes under a shared /ELECTION path, allowing services to elect a primary node, monitor predecessor nodes, and automatically re-elect a new leader when the current one fails, ensuring continuous high availability across distributed applications.
Distributed locks prevent concurrent conflicts over shared resources in distributed systems, and this article explains ZooKeeper’s approach using ordered sequential nodes, illustrating the lock acquisition and release process with command-line examples and a complete Java implementation.
This article explains HBase’s internal architecture, covering the roles of HRegionServer, Client, Zookeeper, Master, RegionServer, the physical storage layout, StoreFile and HFile structures, and the Write-Ahead Log mechanism that ensures data durability and fault tolerance.
This tutorial introduces Apache Mesos, Zookeeper, and Marathon, explains their core components and coordination mechanisms, and provides detailed, image‑rich step‑by‑step instructions for setting up a pseudo‑cluster, deploying Docker containers, and managing tasks through the Mesos and Marathon web interfaces.
This article explains the fundamentals of Apache ZooKeeper, its role as a high‑performance coordination service for distributed applications, and provides a step‑by‑step guide to installing, configuring, and operating a three‑node ZooKeeper ensemble with practical CLI examples.
This article provides a comprehensive, step‑by‑step tutorial for installing Go 1.4, compiling Codis from source, configuring Zookeeper, setting up Codis server, proxy, and dashboard, and deploying a three‑node Codis cluster on Linux, including all necessary environment variables and command examples.
This article provides a comprehensive overview of Apache Flume, covering its design goals, core components, deployment architecture, configuration patterns, and step‑by‑step instructions for integrating Flume with Zookeeper and Kafka to collect and forward massive log data.
This article explains why Kafka introduced high‑availability features after version 0.8, detailing the necessity of data replication and leader election, describing Kafka’s replica distribution algorithm, replication mechanics, acknowledgment requirements, leader‑election strategies, Zookeeper structures, and the broker failover process.
ZooKeeper achieves strong data consistency across its server cluster by implementing the Paxos consensus algorithm, where clients propose changes, servers vote on proposals with monotonically increasing IDs, and a leader coordinates writes to ensure only one operation is committed at a time.
ZooKeeper is an open‑source distributed coordination service that provides high‑performance, reliable primitives such as naming, configuration management, leader election, locks and queues, and its internal design features a leader‑based server cluster, in‑memory hierarchical namespace, and transparent client routing for read/write operations.
Elastic-Job is a Java-based, decentralized distributed job scheduling framework that addresses limitations of existing solutions by providing features such as distributed coordination via Zookeeper, parallel task execution, elastic scaling, centralized management, customizable workflow tasks, and robust non‑functional requirements, with future plans for multi‑language support and enhanced monitoring.
Design a flash‑sale (秒杀) platform that handles millions of concurrent purchase requests without overselling by combining web‑server load shedding, ZooKeeper‑based inventory quotas, message‑queue consumers, stored‑procedure database updates, slot‑based row locking, and hot‑item isolation, all detailed with code examples.
This article introduces Elastic-Job, a Java‑based distributed job scheduling framework from Dangdang, covering its origins, core features such as sharding and elastic scaling, deployment with Zookeeper, best‑practice usage, open‑source development philosophy, future improvements, and detailed Q&A.
This guide explains what zkdash is, provides step‑by‑step installation and setup instructions, details its configuration file options, shows how to integrate with QConf, and highlights important usage notes for managing Zookeeper clusters.
This article explains how large‑scale internet companies replace local service calls with remote procedure calls (RPC), covering Java dynamic proxies, message encoding, serialization, asynchronous response handling, and Zookeeper‑based service registration and discovery to achieve transparent, scalable service communication.
This article explains Apache Kafka's distributed publish‑subscribe design, core components, storage model, ZooKeeper coordination, performance benchmarks against ActiveMQ and RabbitMQ, and provides Java producer and consumer code examples for building high‑throughput messaging applications.
The article explains why Kafka introduced high availability starting with version 0.8, detailing the need for data replication and leader election, describing replica distribution algorithms, replication mechanics, ISR handling, ZooKeeper structures, and the broker failover process to ensure fault‑tolerant streaming.
This guide offers a complete collection of practical examples that demonstrate Curator's usage for Apache ZooKeeper, covering leader election, distributed locks, barriers, counters, caches, queues, and transaction handling, with each chapter independent and source code available on GitHub.
This article explains Hadoop 2.0's High Availability architecture, comparing the NFS and Quorum Journal Manager methods, detailing their principles, failover mechanisms, and practical tips for reliable NameNode redundancy in big‑data deployments.
The article explains why Zookeeper is a crucial component for implementing master‑slave high‑availability architectures in distributed systems, detailing how temporary nodes, sequence numbers, and cluster coordination eliminate single points of failure and enable reliable failover for write services.
