Improving CI/CD Pipeline Speed with Self-Hosted GitLab Runners, Caching, Alpine Images, and Conditional Jobs

As with any continuous integration and continuous deployment platform, speed is critical for developer efficiency.

1. Use Self-Hosted GitLab Runner

GitLab.com provides shared runners for every repository, which is great for a quick start, but the biggest single‑speed gain we observed came from hosting our own runner. The bottleneck was not CPU or RAM but network bandwidth. On a private‑cloud server, network speed increased dramatically, which is especially important for builds and deployments that need to download libraries, dependencies, Docker images, etc., or upload artifacts to remote locations. When the shared GitLab runners become saturated, these stages slow down.

2. Cache Build Dependencies

Storing build dependencies in a private internal repository is much faster than fetching them from the internet. Installing dependencies on every CI job wastes time. Instead, use Docker images that already contain all required dependencies for the CI job. Build caches can be saved using the pipeline cache syntax or via a global cache configuration.

3. Use Alpine CI Images for Builds

Whenever possible, run CI jobs on small Linux distribution images. Alpine Linux is a popular choice, though other lightweight images exist. For example, an Ubuntu image can be 30‑40 times larger than an Alpine image, leading to longer download times. You can also adjust the runner’s image‑pull strategy, such as pre‑downloading images locally and configuring the runner to pull from the local cache when the image is not present.

4. Reduce Unnecessary Job Runs

Run jobs only when relevant files change. To save time, consider using only:changes to conditionally execute jobs. List the directories/files whose modifications should trigger the job, ensuring you include all possible impacts, including shared dependencies. Example:

test-example1:
  script:
    - yarn --cwd apps/example1/ test
  only:
    changes:
      - apps/example1/**/*
      - shared-dependencies/**/*
test-example2:
  script:
    - yarn --cwd apps/example2/ test
  only:
    changes:
      - apps/example2/**/*
      - shared-dependencies/**/*