Boost Compilation Speed with Parallel Make, Tmpfs, ccache, and distcc

Parallel compilation with make

Using the -j option enables concurrent compilation jobs. For a CPU with n logical cores, a typical choice is make -j n (e.g., make -j4 on a four‑core system). This keeps all cores busy without overwhelming the scheduler. Setting the job count far above the core count can cause excessive context switches and may even corrupt shared library archives, so the value should be chosen conservatively.

Eliminating I/O bottlenecks with RAM‑backed filesystems

Compiling generates many temporary object files. Placing these files on a memory‑based filesystem reduces disk latency dramatically. On Linux you can create a dedicated tmpfs mount: mount -t tmpfs tmpfs /mnt -o size=10G or simply use the existing /run/shm directory, which is already a tmpfs mount. Ensure the mount size is sufficient for the expected number of intermediate files.

Object‑file caching with ccache

Install ccache and make the compiler wrappers take precedence in $PATH. A common approach is to create symbolic links in /usr/local/bin that point to the ccache binary:

ln -s /usr/bin/ccache /usr/local/bin/gcc
ln -s /usr/bin/ccache /usr/local/bin/g++
ln -s /usr/bin/ccache /usr/local/bin/cc
ln -s /usr/bin/ccache /usr/local/bin/c++

Because /usr/local/bin is usually searched before /usr/bin, any invocation of gcc, g++, etc., will be intercepted by ccache. Cache statistics can be inspected with:

ccache -s

Distributed compilation with distcc

distcc forwards the preprocessing and compilation steps to one or more remote hosts while the linking step remains on the initiating machine. Requirements:

The source tree must be buildable with make -j (i.e., it supports parallel builds).

All participating machines must have compatible versions of the compiler (same major version and target architecture).

Each remote host runs the distccd daemon and is reachable over the network.

Typical usage involves setting the DISTCC_HOSTS environment variable, e.g.:

export DISTCC_HOSTS="host1/4 host2/4 host3/4"
make -j$(nproc) CC="distcc gcc" CXX="distcc g++"

This distributes up to four jobs to each listed host. The tool automatically handles preprocessing, sending the source to the remote node, and receiving the compiled object file.

Combining the techniques

By running make -j on a tmpfs directory, enabling ccache for object reuse, and optionally adding distcc to spread compilation across multiple machines, developers can achieve substantial reductions in overall build time on Linux systems.