How Alibaba’s Dragonfly P2P Architecture Solves Large‑Scale Docker Image Distribution

Background and Problem

When a container cluster grows, pulling Docker images from a central registry becomes a serious bottleneck. Alibaba’s internal images often reach 3–4 GB because they include large base images, numerous middleware components, and Java binaries. The sheer number of containers (hundreds of thousands) and frequent releases cause the image registry to be overwhelmed, leading to bandwidth saturation and, in some cases, complete service disruption.

In addition, the default gzip compression used for image layers is single‑threaded and CPU‑intensive. When multiple containers decompress layers simultaneously, they compete for CPU resources, causing noticeable business jitter.

Dragonfly (蜻蜓) P2P Architecture

Dragonfly is a CNCF‑based peer‑to‑peer (P2P) distribution system designed to alleviate registry pressure. Its three‑layer architecture consists of a configuration center, super‑nodes, and physical nodes running containers.

Physical nodes request images via the dfget command, which contacts the configuration center to obtain a list of super‑nodes.

Super‑nodes first check their local cache; if the requested chunk is present, they serve it directly to the requester.

If a super‑node lacks the chunk, it fetches the missing part from the image registry in a chunked manner, allowing only failed chunks to be retried.

Because multiple nodes requesting the same data trigger only a single download from the registry, the overall load on the registry is dramatically reduced.

Effectiveness of Dragonfly

Performance graphs show that as concurrent image pulls increase, the traditional pull method’s latency grows steeply, while Dragonfly’s latency rises only slightly, demonstrating its efficiency in large‑scale environments.

Remaining Challenges

Dragonfly does not eliminate the CPU overhead of gzip decompression, nor does it fully address rapid scaling scenarios where containers must be added within minutes during traffic spikes.

Remote‑Disk Architecture

To further reduce image download time, Alibaba introduced a remote‑disk solution. Each image layer is stored on a remote disk with a unique identifier; the container image references this ID. This approach shrinks image metadata to a few kilobytes, enabling downloads in 1–2 seconds.

Since the remote disks reside in the same data center as the physical nodes, read latency is comparable to local SSDs, while still offering the flexibility of network‑based storage.

However, remote‑disk reads consume significant network bandwidth, and any I/O hang on the remote disk can turn containers into zombie processes, harming service availability. The infrastructure cost is also higher due to the need for high‑performance storage.

DADI: Combining P2P, Compression, and Caching

DADI builds on the lessons from Dragonfly and remote‑disk designs. It employs P2P data exchange to share chunks between nodes, uses a fast compression algorithm (e.g., LZ4) to reduce bandwidth, and caches the full image locally on the node after the first pull.

Local caching ensures that subsequent container starts are unaffected by network issues, eliminating the risk of zombie containers caused by remote‑disk I/O failures.

Conclusion

By integrating Dragonfly’s P2P distribution, remote‑disk’s lightweight image references, and DADI’s compression plus local caching, Alibaba achieves a scalable, low‑latency image delivery pipeline that mitigates registry overload, reduces CPU‑induced jitter, and supports rapid container scaling in both on‑premises and cloud environments.