Master GitLab CI: Step‑by‑Step Guide to Runners, .gitlab-ci.yml and Pipelines

Tech from Big Companies – Weekly Updates – Curated Articles

1. What is GitLab CI?

Modern continuous software development requires repeated builds, tests, and deployments, which are tedious but essential for keeping code up‑to‑date. This need leads to CI/CD, and GitLab CI/CD is GitLab's official service that provides automated pipelines to handle these repetitive tasks.

For example, ByteDance's internal tool E*** CI builds on GitLab CI and Codebase CI, adding finer scenario classification and better monorepo support.

2. How to use it?

GitLab CI usage mainly involves two steps

Step 1: Configure a runner

We can think of a GitLab runner as the executor of the .gitlab-ci.yml file. The .gitlab-ci.yml tells the runner what to do.

GitLab runner is not a simple configuration item; it must be deployed, e.g., by running a Docker image that can access the internal network, or you can use shared runners provided by the company.

Use a public shared runner or create your own runner.

Use shared runner:

Just enable the shared runner option and use it directly.

Configure your own runner:

Official documentation for deploying a GitLab runner: Run GitLab Runner in a container | GitLab [1]; Registering runners | GitLab [2]

A. Install Docker

brew install --cask docker

B. Start GitLab runner container

Run a Docker container with the GitLab Runner image.

docker run -d --name gitlab-runner --restart always \
  -v /Users/Shared/gitlab-runner/config:/etc/gitlab-runner \
  -v /var/run/docker.sock:/var/run/docker.sock \
  gitlab/gitlab-runner:latest

Note : On macOS the shared folder should be /Users/Shared/ instead of /srv.

C. Register the runner to GitLab

After starting the container, you need to register it with your GitLab repository using the URL and registration token obtained from the UI.

Replace --url and --registration-token in the command below with the values from the UI. --tag-list defines tags for the runner, used when configuring jobs.

docker run --rm -v /Users/Shared/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner register \
  --non-interactive \
  --executor "docker" \
  --docker-image alpine:latest \
  --url "https://gitlab.com/" \
  --registration-token "wc-Reg7qUpGRB4Lw3q9Y" \
  --description "gitlab-cicd-runner" \
  --tag-list "yehanli,liyehan" \
  --run-untagged="true" \
  --locked="false" \
  --access-level="not_protected"

After registration, you can see the runner listed in Settings → CI/CD.

You can also see the running runner inside Docker:

Cannot display outside Feishu docs.

Step 2: Create .gitlab-ci.yml file

In the root of the monorepo, create a file named .gitlab-ci.yml and push it to the master branch.

git add .gitlab-ci.yml
git commit -m "Add .gitlab-ci.yml"
git push origin master

Note : In older GitLab versions (e.g., 11.4), if .gitlab-ci.yml is missing on master, the GitLab Runner tab may not appear.

In project Settings → CI/CD → General pipelines you can customize the CI config file path; the default is .gitlab-ci.yml.

3. How to write .gitlab-ci.yml ?

First, get familiar with YAML syntax and its differences from JSON.

YAML syntax details: YAML Syntax – Ansible Documentation [3]

YAML syntax:

Array syntax:

{ "array": ["red", "blue", "yellow"] }

Converted to YAML:

array:
  - red
  - blue
  - yellow

Object syntax:

{ "student1": { "name": "小明" }, "array": ["red","blue","yellow"], "student2": { "name": "小明" } }

Converted to YAML:

# I am a comment
student1:
  name: 小明
array:
  - red
  - blue
  - yellow
student2:
  name: 小明

YAML comments use #.

Pipeline stages overview:

The pipeline flow can be divided into the following steps based on the .gitlab-ci.yml configuration:

Configuration before each pipeline job.

Define a single pipeline job (task) and configure it.

Structure diagram:

A single job example (fields explained later):

# This my_job name can be customized
my_job:
  only:
    - master
  tags:
    - yehanli
  script:
    - echo 'job ========= completed'
  stage: build
  retry: 2

GitLab CI keywords:

GitLab CI has many keywords, but only a subset is commonly used. For special configurations see the keyword reference.

We will introduce the common keywords first, then examine a full configuration demo.

script

Scripts to run, defined as an array.

pipeline_job:
  ...
  script:
    - cd <folder>
    - npm install
    - npm build
  ...

before_script

Scripts that run before all jobs, or before a specific job's script.

variables

Variables in GitLab CI can be:

Predefined variables, e.g., CI_COMMIT_BRANCH.

Variables defined in Settings → CI/CD → Variables.

Variables defined directly in .gitlab-ci.yml.

variables:
  TEST_VAR: "All jobs can use this variable's value"

job1:
  variables:
    TEST_VAR_JOB: "Only job1 can use this variable's value"
  script:
    - echo "$TEST_VAR"
  ...

image

The Docker image used as the execution environment, e.g., node, python, java.

Common images: node, python, java, etc.

stages

You can define custom pipeline stages. Jobs in earlier stages must succeed before later stages run.

stages:
  - first_stage
  - second_stage
  - third_stage
  - fourth_stage
  - fifth_stage

If no custom stages are defined, the default stages are .pre → build → test → deploy → .post.

stages:
  - .pre # runs before all stages
  - build
  - test
  - deploy
  - .post # runs after all stages

Note : Jobs in the build stage run in parallel; later stages run sequentially.

stage

Specifies which stage a job belongs to.

cache

Caches files/directories between jobs.

cache:
  key: cache-node-modules
  paths:
    - node_modules

retry

Maximum number of retries for a job (0‑2).

job:
  script: rspec
  retry:
    max: 2
    when: runner_system_failure

only & except

Define when jobs run or do not run.

only: sets job execution timing.

except: sets when job should not run.

Common branch triggers:

Branch name

Trigger

pushes

Triggered on git push for all branches

merge_requests

Triggered when a merge request is created

web

Manually run pipeline via UI

Example using only:

myjob:
  only:
    - master
# equivalent to
myjob:
  only:
    variables:
      - $CI_COMMIT_REF_NAME == "master"

rules:if

Can be used under a job or workflow to conditionally run based on an if statement.

rules:
  - if: $CI_COMMIT_REF_NAME =~ /feature/

Note : only & except and rules:if cannot be used together.

workflow

Used with rules to control the entire pipeline execution.

variables:
  IS_FEATURE: "false"
workflow:
  rules:
    - if: $CI_COMMIT_REF_NAME =~ /feature/
      variables:
        IS_FEATURE: "true"
    - when: always # Run pipeline in other cases

when

Controls job execution based on previous stage results.

on_success (default): runs when previous stages succeeded.

on_failure: runs when any previous stage failed.

always: runs regardless of previous status.

manual: requires manual trigger.

delayed: runs after a delay.

timed rollout 10%:
  stage: deploy
  script: echo 'Rolling out 10% ...'
  when: delayed
  start_in: 30 minutes

tags

Specifies which runner to use for the job.

Important notes

Some keywords like workflow and rules may cause errors if the GitLab version is older than 12.5. For example, only with merge_requests is supported only after version 11.6.

Modular writing

As pipeline jobs increase, putting everything in .gitlab-ci.yml becomes bulky. The solution is to split tasks into multiple modules.

In .gitlab.yml you can include other yml files:

include:
  - '/yml/job_1_install.yml'
  - '/yml/job_2_build.yml'
  - '/yml/job_3_deploy.yml'

4. Demo configuration + run examples

Below we run examples to see how the configuration works.

Pipeline Demo 1 (basic usage)

# .gitlab-ci.yml
# Use node image
image: node:latest

# Custom stages
stages:
  - first_stage
  - second_stage
  - third_stage
  - fourth_stage
  - fifth_stage

# Job 1
job_1_push:
  only:
    - pushes
  tags:
    - yehanli
  script:
    - echo 'job1 ========= completed'
  stage: first_stage

# Job 2
job_2_push:
  only:
    - pushes
  tags:
    - yehanli
  script:
    - echo 'job2 ========= completed'
  stage: third_stage

# Job 3 (manual)
job_3_push:
  only:
    - pushes
  tags:
    - yehanli
  script:
    - echo 'job3 ========= completed'
  stage: fourth_stage
  when: manual

# Job 4 (always)
job_4_push:
  only:
    - pushes
    - tags
  tags:
    - yehanli
  script:
    - echo 'job4 ========= completed'
  stage: fourth_stage
  when: always

# Job 5 (web trigger)
job_5_web:
  only:
    - web
  tags:
    - yehanli
  script:
    - echo 'job5 ========= completed'
  stage: fifth_stage

Job 3 requires manual execution.

Outputs of the first four jobs are shown.

Job 5 can be triggered by clicking Run pipeline, selecting a branch, and then running.

Click Run pipeline in the pipeline UI.

Select the branch and click Run pipeline.

Pipeline Demo 2 (detailed example)

# .gitlab-ci.yml
image: node:latest
before_script:
  - echo '====== Run before all jobs ======'

variables:
  IS_EXPERIENCING_MERGING: "false"
  IS_ON_MYBRANCH: "false"

workflow:
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
      variables:
        IS_EXPERIENCING_MERGING: "true"
    - if: $CI_COMMIT_REF_NAME =~ /myBranch/
      variables:
        IS_ON_MYBRANCH: "true"
    - if: $CI_COMMIT_REF_NAME =~ /.*/

cache:
  key: cache-node-modules
  paths:
    - node_modules

stages:
  - first_stage
  - second_stage
  - third_stage
  - fourth_stage
  - fifth_stage

include:
  - '/yml/job_1_install.yml'
  - '/yml/job_2_build.yml'
  - '/yml/job_3_deploy.yml'
  - '/yml/job_4_mybranch.yml'
  - '/yml/job_5_beforeMR.yml'

Example of an included job file ( /yml/job_1_install.yml):

# job_1_install.yml
# install stage
job_1_install:
  only:
    - master
  tags:
    - yehanli
  before_script:
    - echo '========== job1 script before execution =========='
    - npm install
  script:
    - echo 'job1 ========= completed'
  stage: first_stage
  retry: 2

Example of an included job file ( /yml/job_2_build.yml):

# job_2_build.yml
# build stage
job_2_build:
  only:
    - master
  tags:
    - yehanli
  script:
    - npm run build
    - echo 'job2 ========= completed'
  stage: second_stage
  retry: 2

Example of an included job file ( /yml/job_3_deploy.yml):

# job_3_deploy.yml
# deploy stage
job_3_deploy:
  image: docker
  only:
    - master
  tags:
    - yehanli
  script:
    - docker build -t reactimages .
    - if [ $(docker ps -aq --filter name=react-container) ]; then docker rm -f react-container; fi
    - docker run -d -p 8000:80 --name react-container reactimages
    - echo 'job3 ========= completed'
  stage: third_stage
  when: always

Example of an included job file ( /yml/job_4_mybranch.yml):

# job_4_mybranch.yml
job_4_mybranch:
  only:
    - myBranch
  tags:
    - yehanli
  script:
    - echo 'is it on myBranch?' $IS_ON_MYBRANCH
    - echo 'job4 ========= completed'
  stage: fourth_stage
  retry: 2

Example of an included job file ( /yml/job_5_beforeMR.yml):

# job_5_beforeMR.yml
job_5_beforeMR:
  rules:
    - if: '$CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"'
  tags:
    - yehanli
  script:
    - echo 'Is experiencing before‑merge?' $IS_EXPERIENCING_MERGING
    - echo '========== job_5_beforeMR completed =========='
  stage: fifth_stage
  retry: 2

job_5_afterMR:
  rules:
    - if: '$CI_COMMIT_BRANCH == "master"'
  tags:
    - yehanli
  script:
    - echo 'Is experiencing merging?' $IS_EXPERIENCING_MERGING
    - echo '========== job_5_afterMR completed =========='
  stage: fifth_stage
  retry: 2

Dockerfile example used in a job:

FROM node:latest as builder
WORKDIR /app
COPY package.json ./
COPY package-lock.json ./
RUN npm install --registry=https://bnpm.byted.org
COPY . ./
RUN npm run build
FROM nginx:latest
COPY --from=builder /app/build /usr/share/nginx/html

Reference materials

Run GitLab Runner in a container | GitLab: https://docs.gitlab.com/runner/install/docker.html

Registering runners | GitLab: https://docs.gitlab.com/runner/register/index.html#docker

YAML Syntax – Ansible Documentation: https://docs.ansible.com/ansible/latest/reference_appendices/YAMLSyntax.html

Keyword reference for the .gitlab-ci.yml file: https://docs.gitlab.com/ee/ci/yaml/

GitLab CI/CD documentation: https://docs.gitlab.com/ee/ci/yaml/

Predefined variables reference | GitLab: https://docs.gitlab.com/ee/ci/variables/predefined_variables.html

Custom CI/CD variable in .gitlab-ci.yml: https://docs.gitlab.com/ee/ci/variables/index.html#create-a-custom-cicd-variable-in-the-gitlab-ciyml-file