Improving CI/CD Pipeline Speed with Self‑Hosted GitLab Runners, Caching, Alpine Images, and Conditional Jobs

1. Use Self‑Hosted GitLab Runner

GitLab.com provides shared runners for each repository, which is convenient for a quick start, but we found the biggest speed gain comes from hosting our own runners. For us, the bottleneck was not CPU or RAM but the network. On a private‑cloud server, network speed increased dramatically. Network speed is especially important for builds and deployments because builds need to download libraries, dependencies, Docker images, etc., and deployments need to upload artifacts. When the shared GitLab runners are saturated, these stages become slow.

2. Cache Build Dependencies

Storing build dependencies in a private internal repository is much faster than fetching them from the Internet. Installing dependencies on every CI job wastes time. Instead, you should use Docker images that already contain all required dependencies for CI jobs. Build caches can be saved using the pipeline cache keyword or a global cache.

3. Use Alpine CI Images

Use small Linux distribution images to run CI jobs whenever possible. Alpine Linux is a popular choice, though other options exist. The reason is that using a large distribution such as Ubuntu can result in images that are 30‑40 times larger, leading to longer download times. You can also pre‑download images to the runner and configure the runner’s image‑pull policy to “pull from remote only if not present locally”.

4. Reduce Unnecessary Job Execution

Run jobs only when relevant files change. To save time, consider using the only:changes keyword to conditionally run jobs. List the directories/files whose modifications should trigger the job, including shared dependencies. Example:

test-example1:
  script:
    - yarn --cwd apps/example1/ test
  only:
    changes:
      - apps/example1/**/*
      - shared-dependencies/**/*
test-example2:
  script:
    - yarn --cwd apps/example2/ test
  only:
    changes:
      - apps/example2/**/*
      - shared-dependencies/**/*