How to Build a Full CI/CD Pipeline for Python with GitLab, Docker, and SonarQube

Taking advantage of a migration to a private GitLab instance, the author implements continuous integration for a Python backend, covering unit testing, static analysis, and API testing.

GitLab CI Basic Configuration

Enable CI/CD for the project and register a GitLab runner.

GitLab Runner

Run the runner in a Docker container to keep resource usage low.

sudo docker pull gitlab/gitlab-runner

sudo docker run -d --name gitlab-runner --restart always \
    -v /srv/gitlab-runner/config:/etc/gitlab-runner \
    -v /var/run/docker.sock:/var/run/docker.sock \
    gitlab/gitlab-runner:latest

Register the runner inside the container:

sudo docker exec -it gitlab-runner gitlab-ci-multi-runner register

Local Executor Image

Configure the runner to use only local images:

[runners.docker]
pull_policy = never  # default is always

Optional host mappings can be added under the same section: extra_hosts = ["hostname:ip"] Pre‑install Python dependencies in the executor image to speed up tests:

COPY requirement.txt .
RUN pip install -r requirement.txt

.gitlab-ci.yml

Unit Test Stage

UnitTest:
  stage: unittest
  script:
    - pip install nose
    - python -m runtest xtest

SonarQube Analysis

Deploy SonarQube with Docker‑Compose (PostgreSQL backend) and use a custom sonar‑scanner image that includes pylint for Python analysis.

git clone https://github.com/ityoung/sonarqube-docker.git
pip install docker-compose
cd sonarqube-docker
docker-compose up

Build the scanner image:

git clone https://github.com/ityoung/sonar-scanner-docker.git
cd sonar-scanner-docker
docker build -t sonar-scanner .

Add the scanner command to the CI job and configure Sonar parameters via variables.

Service Containers

Pull Redis and MongoDB images locally and declare them as services in the pipeline:

services:
  - redis:2.8
  - mongo:3.2

Update the application’s configuration to point to the service names redis and mongo.

Deploy Stage

Deploy_TestServer:
  stage: deploy
  script: echo "deploy"

API Test Stage

API_Test:
  stage: apitest
  script:
    - cd /
    - git clone https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.xxx.com/yx/apitest.git
    - cd apitest
    - pip install -r requirements.txt
    - python runtest.py

Full .gitlab-ci.yml Example

image: pro1_executor
stages:
  - unittest
  - analyze
  - deploy
  - apitest
variables:
  SONAR_HOST: "http://192.168.0.29:9000"
  SONAR_PROJ: "pro_1"
  SONAR_LOGIN: "b3135dd602b61ce7ff5f4202a3ec2ec0865fa7f5"
services:
  - redis:3
  - mongo:3.2
UnitTest:
  stage: unittest
  script:
    - pip install nose
    - python -m runtest xtest
Sonar_Preview:
  image: sonar-scanner
  stage: analyze
  script:
    - sonar-scanner
    -Dsonar.analysis.mode=preview
    -Dsonar.gitlab.commit_sha=$CI_BUILD_REF
    -Dsonar.gitlab.ref_name=$CI_BUILD_REF_NAME
    -Dsonar.gitlab.project_id=$CI_PROJECT_ID
    -Dsonar.projectKey=$SONAR_PROJ
    -Dsonar.sources=.
    -Dsonar.host.url=$SONAR_HOST
    -Dsonar.login=$SONAR_LOGIN
    except:
      - master
Deploy_TestServer:
  stage: deploy
  script: echo "deploy"
API_Test:
  stage: apitest
  script:
    - cd /
    - git clone https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.xxx.com/yx/apitest.git
    - cd apitest
    - pip install -r requirements.txt
    - python runtest.py

Reference: "how to access multiple repositories in CI build?" – Stack Overflow.