Mastering Continuous Integration and Deployment with GitLab CI and Docker

Continuous Integration (CI)

Continuous Integration means merging code changes into the main branch frequently (often multiple times a day). The main benefits are rapid error detection and preventing large branch divergences that make later integration difficult. Martin Fowler noted that CI does not eliminate bugs but makes them easy to find and fix.

Related concepts: Continuous Delivery (CD) and Continuous Deployment

Continuous Delivery extends CI by delivering new software versions to a quality team or users for review; if approved, the code proceeds to production. It emphasizes that the software can be released at any time. Continuous Deployment goes one step further: after review, the code is automatically deployed to production, requiring fully automated testing, building, and deployment steps.

Typical CI workflow

Commit : Developer pushes code to the repository, triggering the pipeline.

First‑round tests : Automated unit tests (and optionally integration or end‑to‑end tests) run immediately.

Build : After passing the first tests, the code is built (dependencies installed, assets compiled, etc.).

Second‑round tests : Comprehensive testing, including integration and end‑to‑end tests, runs before deployment.

Deploy : The built artifact is packaged and deployed to a production‑like environment, then to production.

Rollback : If issues arise, the previous version is restored by switching symbolic links.

GitLab CI basics

GitLab CI uses a .gitlab-ci.yml file to define pipelines, stages, and jobs. A pipeline represents a single build task and can contain multiple stages such as install, test, build, and deploy. Jobs are the individual tasks that run within a stage. Stages are executed sequentially; jobs in the same stage run in parallel. If any job fails, the pipeline stops.

GitLab Runner

GitLab Runner is the agent that executes the jobs defined in a pipeline. Installing the runner on a separate machine or as a Docker container isolates build resources from the GitLab server, preventing performance degradation.

Docker‑based Runner setup (docker‑compose)

version: '2'
services:
  gitlab:
    image: twang2218/gitlab-ce-zh:10.5
    restart: always
    hostname: '10.3.50.160'
    container_name: gitlab
    environment:
      TZ: 'Asia/Shanghai'
      GITLAB_OMNIBUS_CONFIG: |
        external_url 'http://10.3.50.160:8080'
        gitlab_rails['gitlab_shell_ssh_port'] = 2222
        unicorn['port'] = 8888
        nginx['listen_port'] = 8080
    ports:
      - '8080:8080'
      - '8443:443'
      - '2222:22'
    volumes:
      - /etc/localtime:/etc/localtime
      - ./conf:/etc/gitlab
      - ./data/logs:/var/log/gitlab
      - ./data/data:/var/opt/gitlab
  gitlab-runner:
    image: gitlab/gitlab-runner
    restart: always
    hostname: gitlab-runner
    container_name: gitlab-runner
    extra_hosts:
      - git.imlcs.top:10.3.50.160
    depends_on:
      - gitlab
    volumes:
      - /etc/localtime:/etc/localtime
      - ./runner:/etc/gitlab-runner
      - /var/run/docker.sock:/var/run/docker.sock

Registering a Runner

Method 1 (non‑interactive):

docker exec -it gitlab-runner gitlab-runner register -n \
  --url http://10.3.50.160:8080/ \
  --registration-token cpR4sgBCsZ-TJUpJVz9t \
  --description "dockersock" \
  --docker-privileged=true \
  --docker-pull-policy="if-not-present" \
  --docker-image "docker:latest" \
  --docker-volumes /var/run/docker.sock:/var/run/docker.sock \
  --docker-volumes /root/m2:/root/.m2 \
  --executor docker

Method 2 (interactive):

docker exec -it gitlab-runner gitlab-runner register
# Follow the prompts to provide URL, token, description, tags, etc.

Sample .gitlab-ci.yml files

Simple Spring Boot project

image: docker-maven:alpine
services:
  - redis:3-alpine
stages:
  - build
build app:
  stage: build
  script:
    - mvn -Dmaven.test.skip=true clean package docker:build

Multi‑stage pipeline (Node.js example)

stages:
  - install_deps
  - test
  - build
  - deploy_test
  - deploy_production
cache:
  key: ${CI_BUILD_REF_NAME}
  paths:
    - node_modules/
    - dist/
install_deps:
  stage: install_deps
  only:
    - develop
    - master
  script:
    - npm install
test:
  stage: test
  only:
    - develop
    - master
  script:
    - npm run test
build:
  stage: build
  only:
    - develop
    - master
  script:
    - npm run clean
    - npm run build:client
    - npm run build:server
deploy_test:
  stage: deploy_test
  only:
    - develop
  script:
    - pm2 delete app || true
    - pm2 start app.js --name app
deploy_production:
  stage: deploy_production
  only:
    - master
  script:
    - bash scripts/deploy/deploy.sh

Runner management commands

gitlab-ci-multi-runner unregister --name "my-runner"

gitlab-ci-multi-runner list

Dockerfile for a Java service

FROM openjdk:8-jre
ENV APP_VERSION 1.0.0-SNAPSHOT
ENV DOCKERIZE_VERSION v0.6.1
RUN wget https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz \
    && tar -C /usr/local/bin -xzvf dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz \
    && rm dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz
RUN mkdir /app
COPY myshop-service-user-provider-$APP_VERSION.jar /app/app.jar
ENTRYPOINT ["dockerize","-timeout","5m","-wait","tcp://192.168.10.131:3306","java","-Djava.security.egd=file:/dev/./urandom","-jar","/app/app.jar"]
EXPOSE 8501

CI pipeline for the service provider

stages:
  - build
  - push
  - run
  - clean
build:
  stage: build
  script:
    - /usr/local/maven/apache-maven-3.5.3/bin/mvn clean package
    - cp target/myshop-service-user-provider-1.0.0-SNAPSHOT.jar docker
    - cd docker
    - docker build -t 192.168.10.133:5000/myshop-service-user-provider:v1.0.0 .
push:
  stage: push
  script:
    - docker push 192.168.10.133:5000/myshop-service-user-provider:v1.0.0
run:
  stage: run
  script:
    - cd docker
    - docker-compose down
    - docker-compose up -d
clean:
  stage: clean
  script:
    - docker image prune -f

docker‑compose.yml for the provider

version: '3.1'
services:
  myshop-service-user-provider:
    image: 192.168.10.133:5000/myshop-service-user-provider:v1.0.0
    container_name: myshop-service-user-provider
    ports:
      - 8501:8501
      - 22222:22222
      - 20881:20881
networks:
  default:
    external:
      name: dubbo

CI pipeline for the service consumer

stages:
  - build
  - push
  - run
  - clean
build:
  stage: build
  script:
    - /usr/local/maven/apache-maven-3.5.3/bin/mvn clean package
    - cp target/myshop-service-user-consumer-1.0.0-SNAPSHOT.jar docker
    - cd docker
    - docker build -t 192.168.10.133:5000/myshop-service-user-consumer:v1.0.0 .
push:
  stage: push
  script:
    - docker push 192.168.10.133:5000/myshop-service-user-consumer:v1.0.0
run:
  stage: run
  script:
    - cd docker
    - docker-compose down
    - docker-compose up -d
clean:
  stage: clean
  script:
    - docker image prune -f

docker‑compose.yml for the consumer

version: '3.1'
services:
  myshop-service-user-consumer:
    image: 192.168.10.133:5000/myshop-service-user-consumer:v1.0.0
    container_name: myshop-service-user-consumer
    ports:
      - 8601:8601
      - 8701:8701
networks:
  default:
    external:
      name: my_net

Deploying common modules to Nexus

cd ..
cd myshop-dependencies
mvn deploy
cd ..
cd myshop-commons
mvn deploy
# repeat for each additional module as needed