Implementing a Python CI/CD Pipeline with GitLab CI

In May, the GitLab CI tutorial was updated to include a Python delivery pipeline demo; this article provides a generic example that readers can adapt and extend for their own projects.

Typical steps for releasing a Python project include installing dependencies via pip, compiling source files to .pyc, running JUnit unit tests, performing code scanning, building a Docker image, and deploying to Kubernetes.

The Dockerfile used for the Flask web app is shown below:

FROM python:3.7
# copy all files
RUN mkdir hello
COPY . /hello
WORKDIR /hello

# install required libraries
RUN pip install Flask -i https://mirrors.aliyun.com/pypi/simple/  --trusted-host mirrors.aliyun.com
RUN pip install Flask_Script -i https://mirrors.aliyun.com/pypi/simple/  --trusted-host mirrors.aliyun.com

EXPOSE 5000

CMD ["python", "run.py"]

The GitLab CI pipeline configuration defines global variables, job controls, and images, then sets up a build job that installs requirements, extends a shared build template, and prepares the environment for further stages such as testing, code analysis, image building, and Kubernetes deployment.

include:
  - project: 'cidevops/cidevops-newci-service'
    ref: master
    file: 'templates/default-pipeline.yml'

variables:
  ## Global configuration
  GIT_CLONE_PATH: ${CI_BUILDS_DIR}/builds/${CI_PROJECT_NAMESPACE}/${CI_PROJECT_NAME}/${CI_PIPELINE_ID}
  GIT_CHECKOUT: "false"

  ## Job control
  RUN_PIPELINE_BUILD:   "yes"
  RUN_PIPELINE_TEST:    "no"
  RUN_CODE_ANALYSIS:    "yes"
  RUN_BUILD_IMAGE:      "yes"
  RUN_DEPLOY_ARTIFACTS: "no"
  RUN_DEPLOY_K8S:       "yes"

  ## Dependency images
  BUILD_IMAGE: "python:3.7"
  CURL_IMAGE: "curlimages/curl:7.70.0"
  SONAR_IMAGE: "sonarsource/sonar-scanner-cli:latest"
  KUBECTL_IMAGE: "lucj/kubectl:1.17.2"

  ## Build and test parameters
  PIP_CACHE_DIR: "/home/gitlab-runner/ci-build-cache/pip-cache"
  BUILD_SHELL: "python -m compileall . "
  TEST_SHELL : ' '
  JUNIT_REPORT_PATH: 'target/surefire-reports/TEST-*.xml'

  ## Code analysis parameters
  SONAR_SOURCE_DIR : "."
  SONAR_SERVER_URL: "http://192.168.1.200:30090"
  SONAR_SERVER_LOGIN: "ee2bcb37deeb6dfe3a07fe08fb529559b00c1b7b"
  SONAR_SCAN_ARGS: "-Dsonar.sources=${SONAR_SOURCE_DIR} "

  ## Docker image build parameters
  CI_REGISTRY: 'registry.cn-beijing.aliyuncs.com'
  CI_REGISTRY_USER: 'xxxxx'
  IMAGE_NAME: "${CI_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_REF_NAME}-${CI_COMMIT_SHORT_SHA}"
  DOCKER_FILE_PATH: "./Dockerfile"

  ## Deployment to Kubernetes
  APP_NAME: "$CI_PROJECT_NAME"
  CONTAINER_PORT: "5000"
  NAMESPACE: "$CI_PROJECT_NAME-$CI_PROJECT_ID-$CI_ENVIRONMENT_SLUG"
  ENV_URL: "${ENV_NAME}.${CI_PROJECT_NAMESPACE}.${CI_PROJECT_NAME}.devops.com/hello"

build:
  before_script:
    - "pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/  --trusted-host mirrors.aliyun.com "
  extends: .build

Experimental screenshots illustrate the pipeline execution and deployment results.