Distributed Compilation and Build Caching for Large iOS Projects

In recent years, DevOps pipelines have become the industry standard for software development, and the efficiency of these pipelines directly determines team productivity. For large projects, compilation often dominates pipeline time, with some builds exceeding 30 minutes.

ByteDance's iOS projects keep build times under five minutes by using an internal distributed compilation and caching solution. The article first analyzes the bottlenecks of large‑scale compilation and then presents the solution.

Distributed Compilation

Compilation tasks can be split into parallel sub‑tasks. For example, C, C++, and ObjC source files (.c, .cc, .m) are compiled into object files (.o) and then linked. The parallelism is limited by the number of CPU cores; a single PC provides 4‑12 cores, while servers can offer 24‑96 cores, still insufficient for projects with tens of thousands of files.

By distributing compilation across many machines, a "super" computer is formed, increasing total CPU resources and reducing overall build time.

Compilation Cache

Most compilation sub‑tasks have been executed before. A central cache stores the results of previous tasks, indexed by a task summary. When a new task arrives, the system first queries the cache; if a match is found, the compiled artifact is downloaded, avoiding redundant work.

Core Idea

The core idea is simple: compute the list of files a compilation command needs, send the list and command to remote machines, execute the command, and finally retrieve the compiled artifacts.

Dependency Analysis

Accurate dependency analysis is essential for distributed compilation. The pre‑processor expands directives such as #include and #import . For example, the compiler processes a line like #import Car.h by searching include paths (provided by -I or -isystem ) and replacing the directive with the file's contents.

Clang offers flags to extract dependencies: -M , -MM , -MMD , etc. These flags generate depfiles containing system and user headers.

Fast Dependency Analysis

ByteDance's solution combines dependency caching with analysis. If a previous compilation in the same directory exists, the cache is consulted first; only on a miss does full analysis run, achieving sub‑50 ms latency.

Optimization Techniques

Inverted Index : Instead of scanning all include directories for each #import , an inverted index maps header names to directories, reducing file‑system lookups from millions to a few hash lookups.

Cross‑Task Cache : For iOS projects using Xcode and CocoaPods, many compilation commands share identical search paths. By hashing the search‑path list, the system reuses directive results across tasks, dramatically cutting analysis time.

Index Cache : Building the inverted index itself can be costly. By caching the index when search paths and .hmap files remain unchanged, additional 20 ms per analysis is saved.

Summary

Distributed compilation and build caching are the two main levers to improve large‑scale build efficiency. ByteDance's solution builds on the open‑source goma framework, adding iOS‑specific optimizations such as cross‑task directive caching and inverted indexing, achieving compile‑time reductions from minutes to seconds.