Optimizing Large Docker Images for iQIYI Video Transcoding: Differential Build, On‑Demand Download, Deduplication, and Distribution Strategies

The iQIYI video production team presents a set of optimization techniques for large Docker images used in its video transcoding service, leveraging Docker's layered image technology to reduce image size and improve build speed without requiring changes from developers.

Background : The transcoding service runs distributed tasks in containers and relies on Docker images for version release and deployment. Frequent iterations have caused the image size to grow to about 18 GB, leading to significant storage, distribution, and pull overhead. Traditional Docker layering caches unchanged layers, but monolithic images cannot be efficiently optimized using this mechanism.

The all‑in‑one image includes codecs, business code, assets, models, and other tools, all of which change frequently. This results in large image size, long build times, and high bandwidth consumption when deploying to thousands of nodes.

Solution 1 – Differential Build : Instead of rebuilding unchanged layers, a new layer is created on top of an existing base image that contains only the incremental files. Only this new layer needs to be distributed. The process involves selecting a Base image, launching a temporary container, mounting the new version's files, computing file hash differences, writing changed files into a new layer, and recording the new layer's version. Subsequent builds reuse the latest image as the Base, stacking only diff layers. Git can provide file diffs to avoid recursive hash calculations.

Solution 2 – On‑Demand Download : Resources that are not needed for every execution (e.g., GPU‑only libraries) are removed from the base image and downloaded on demand. When a container requests such a resource, the system checks a host‑side cache, downloads the missing files from a remote repository, verifies them via MD5, and creates a soft link inside the container. This reduces image size, build time, and bandwidth usage.

Solution 3 – Image Deduplication : A coverage test identifies which dynamic libraries are actually required by the binaries. Unused libraries installed via the Dockerfile are removed, shrinking the image further.

Solution 4 – Node Pre‑warming & Gray Release : After CI/CD validation, nodes are pre‑warmed by pulling a small percentage (e.g., 10 %) of the new image layers gradually, preventing bandwidth spikes. Gray release combined with pre‑warming reduces peak bandwidth consumption by more than 85 %.

Solution 5 – Dragonfly : The open‑source P2P image distribution system Dragonfly accelerates large‑scale image delivery, saving network bandwidth at the cost of increased outbound traffic on the host machines.

Conclusion : Multi‑dimensional optimizations reduced the image size from 18 GB to 9 GB and cut build time from 30 minutes to under 5 minutes, lowering storage and distribution costs while remaining transparent to developers. Future work will continue to further shrink image volumes.