How to Build Fast, Debuggable Docker Images and Automate CI/CD for Python Projects

Every project—whether a web application, data‑science script, or AI service—can benefit from a well‑configured CI/CD pipeline, Docker images, and code‑quality tools like CodeClimate or SonarCloud. This guide shows how to add these components to a Python project.

Debuggable Development Docker Container

To make a development image easy to debug, we start from python:3.8.1-buster and install common tools (bash, vim, netcat, wget, cat, find, grep). The image is intentionally heavyweight because it is used only for development, and we lock the Python and Debian versions to avoid breakage.

As an alternative, an Alpine‑based image can be used, but it relies on musl libc instead of glibc, which may cause compatibility issues. Multi‑stage builds cache layers to avoid repeatedly downloading gcc and other dependencies listed in requirements.txt.

# dev.Dockerfile
FROM python:3.8.1-buster AS builder
RUN apt-get update && apt-get install -y --no-install-recommends python3-venv gcc libpython3-dev \
    && python3 -m venv /venv && /venv/bin/pip install --upgrade pip
FROM builder AS builder-venv
COPY requirements.txt /requirements.txt
RUN /venv/bin/pip install -r /requirements.txt
FROM builder-venv AS tester
COPY . /app
WORKDIR /app
RUN /venv/bin/pytest
FROM martinheinz/python-3.8.1-buster-tools AS runner
COPY --from=tester /venv /venv
COPY --from=tester /app /app
WORKDIR /app
ENTRYPOINT ["/venv/bin/python3", "-m", "blueprint"]
USER 1001
LABEL name={NAME}
LABEL version={VERSION}

The final runner image is built after three intermediate images: builder (installs gcc and creates a virtual environment), builder-venv (installs Python dependencies), and tester (runs tests). The runner image copies the tested virtual environment and application, sets a non‑root user, and defines the entrypoint.

Production‑Optimized Docker Container

For production we prefer a small, secure image based on Google’s Distroless project. Distroless images contain only the minimal runtime without shells or package managers, reducing attack surface and avoiding CVE noise. The production Dockerfile switches the base image to gcr.io/distroless/python3-debian10, shrinking the image from ~1 GB to ~100 MB.

# prod.Dockerfile
# 1. Change builder image
FROM debian:buster-slim AS builder
# ...
# 17. Switch to Distroless image
FROM gcr.io/distroless/python3-debian10 AS runner
# ... Rest of the Dockerfile

Distroless images lack a shell, making debugging harder; therefore a debug variant is provided, allowing you to exec into the container with

docker run --entrypoint=sh -ti gcr.io/distroless/python3-debian10:debug

.

All Operations with a Single Makefile Command

Makefile targets automate the workflow. make build-dev builds the development image, make build-prod VERSION=1.0.0 builds the production image, make shell launches an interactive shell in the dev container, and make push VERSION=0.0.2 pushes the production image to GitHub Package Registry.

# Example Makefile snippets
MODULE := blueprint
REGISTRY ?= docker.pkg.github.com/martinheinz/python-project-blueprint
IMAGE := $(REGISTRY)/$(MODULE)
TAG := $(shell git describe --tags --always --dirty)

build-dev:
	@echo "Building Development image with labels:"
	@sed -e 's|{NAME}|$(MODULE)|g' -e 's|{VERSION}|$(TAG)|g' dev.Dockerfile | docker build -t $(IMAGE):$(TAG) -f- .

build-prod:
	@echo "Building Production image with labels:"
	@sed -e 's|{NAME}|$(MODULE)|g' -e 's|{VERSION}|$(VERSION)|g' prod.Dockerfile | docker build -t $(IMAGE):$(VERSION) -f- .

shell: build-dev
	docker run -ti --rm --entrypoint /bin/bash -u $$(id -u):$$(id -g) $(IMAGE):$(TAG)

push: build-prod
	docker push $(IMAGE):$(VERSION)

CI/CD with GitHub Actions

Two workflow files are created under .github/workflows: build-test.yml runs on every push and executes make build-dev and the test/lint jobs; push.yml triggers only when a tag is pushed, logs into the Docker registry using a secret token, and runs make push with the tag as the image version.

# build-test.yml (excerpt)
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v1
      - name: Run Makefile build for Development
        run: make build-dev

  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v1
      - uses: actions/setup-python@v1
        with:
          python-version: '3.8'
      - name: Install Dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt
      - name: Run Tests
        run: make test
      - name: Install Linters
        run: |
          pip install pylint flake8 bandit
      - name: Run Linters
        run: make lint

# push.yml (excerpt)
on:
  push:
    tags:
      - '*'
jobs:
  push:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v1
      - name: Set env
        run: echo ::set-env name=RELEASE_VERSION::$(echo ${GITHUB_REF:10})
      - name: Log into Registry
        run: echo "${{ secrets.REGISTRY_TOKEN }}" | docker login docker.pkg.github.com -u ${{ github.actor }} --password-stdin
      - name: Push to GitHub Package Registry
        run: make push VERSION=${{ env.RELEASE_VERSION }}

Code Quality with CodeClimate and SonarCloud

Additional steps are added to the CI workflow to upload coverage reports to CodeClimate and run the SonarCloud scanner. Environment variables are set to capture the current Git branch, and the appropriate secret tokens are used for authentication.

# CodeClimate and SonarCloud steps (excerpt)
- name: Send report to CodeClimate
  run: |
    export GIT_BRANCH="${GITHUB_REF/refs\/heads\//}"
    curl -L https://codeclimate.com/downloads/test-reporter/test-reporter-latest-linux-amd64 > ./cc-test-reporter
    chmod +x ./cc-test-reporter
    ./cc-test-reporter format-coverage -t coverage.py coverage.xml
    ./cc-test-reporter upload-coverage -r "${{ secrets.CC_TEST_REPORTER_ID }}"
- name: SonarCloud scanner
  uses: sonarsource/sonarcloud-github-action@master
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}

Conclusion

With the tools, configurations, and scripts described above, you can fully automate building, testing, linting, and deploying a Python project using Docker, Make, and GitHub Actions.