Design and Implementation of ZZLock Distributed Lock Service Based on Etcd

In the Zhezhu e‑commerce platform, critical operations such as price changes and payments must be coordinated in a distributed environment to guarantee atomicity, eventual consistency, and correct business flow. ZZLock was created to serialize these operations, handling tens of millions of lock requests daily with an average latency of 1.34 ms.

1. Requirement Analysis identified core features: re‑entrancy, reliable mutual exclusion, blocking/non‑blocking APIs, server‑side TTL, and non‑intrusive monitoring.

2. Selection Design evaluated four common solutions:

Database lock tables – lack TTL and high‑cost cleanup.

Redis setnx + expire – good throughput but cannot guarantee strong consistency during master‑slave failover.

Zookeeper (Curator) – provides consistency via ZAB but suffers from limited scalability and complex lease renewal.

Etcd – offers Raft‑based strong consistency and native TTL, though client libraries were immature at the time.

A comparison table summarises consistency algorithms, CAP properties, high‑availability models, interface types, and implementation details for each option.

2.2 Etcd Feasibility explains the Etcd CAS API (/v2/keys/{path}) and four additional requirements: client‑side re‑entrancy binding, spin‑based blocking lock, node health‑check tasks, and integration with the internal monitoring platform.

2.3 Architecture Design shows that a custom Etcd client combined with ZZLock offers a controllable feature set, illustrated by an architecture diagram (image).

2.4 Logical Flow describes the lock acquisition and release sequence, with the client handling all Etcd interactions and reporting metrics to the ZZMonitor platform (image).

3. Development Implementation

3.1 Code Implementation – The main classes include a singleton ZZLockClient, an EtcdLock implementation, and a LockMonitor. After acquiring a lock, a renew thread performs asynchronous lease renewal.

3.2 Code Integration – Usage is straightforward:

try(ZZLock lock = client.newLock("lockName")){
    if(lock.acquire(10)) {
        // 成功获得锁
    }
}

The lock implements Closeable, so the try‑with‑resources block automatically releases the lock.

3.3 Monitoring & Operations – No extra configuration is required; metrics are automatically sent to the internal ZZMonitor platform, which visualises lock usage and alerts on anomalies (image).

4. Special Scenarios

Idempotency – Distributed locks do not solve idempotency; callers must ensure business logic is idempotent, especially for scheduled tasks and rapid lock turnover.

Lease Renewal – Short lease times can cause premature expiration; extend lease duration and ensure the renewal thread has sufficient time.

Etcd Internal Issues – Leader election pauses and Raft log inconsistencies can raise ZZLockException, requiring retry or compensation logic.

5. Summary – ZZLock successfully meets internal requirements using Etcd v2, with potential migration to Etcd v3 + gRPC for sub‑millisecond latency. In environments where ultra‑high availability is needed, a Redis‑based implementation may be viable if strong consistency is not mandatory.

The article also references Martin Kleppmann’s analysis of distributed locking and the limitations of the Redlock algorithm.