Mastering Kubernetes‑Based DevOps: End‑to‑End CI/CD Pipeline with Jenkins, Helm, and Docker

Introduction

The author, a .NET senior developer, shares a hands‑on DevOps practice built on Kubernetes, aiming to help readers understand the whole CI/CD pipeline rather than just copying code.

Preparation

Hardware resources : Test environment – CentOS 7.4+, three VMs (4 CPU + 8 GB + 50 GB) with internal network; Production – Tencent Cloud server CentOS 7.6 (2 CPU + 4 GB) and a CentOS 7.5 node.

Network : Internet access is required to pull images from external registries.

Git repository : https://gitee.com/Gao06/hello-world.git Reference material : Several Docker/Kubernetes/Jenkins video tutorials and books such as "Kubernetes 权威指南" and "阿里巴巴 DevOps 实践手册".

Work Plan

Jenkins installation (Docker)

Kubernetes cluster installation

Helm installation

Baget (private NuGet) installation

Harbor registry installation

SonarQube installation (optional)

Write Kubernetes YAML files

Adjust Dockerfile build context

Write Jenkinsfile pipeline

Testing and adjustments

Software Versions

Docker 19.03.12

Docker‑Compose 1.26.0

Helm v3.2.1 (image dtzar/helm‑kubectl:latest)

Kubernetes server v1.16.3‑tke.9

PostgreSQL (latest)

SonarQube (latest)

SonarScanner ( nosinovacao/dotnet‑sonar:latest)

Baget ( loicsharma/baget:latest)

Jenkins ( jenkinsci/blueocean:latest)

Deployment Architecture

Two servers are used: one (IP 10.129.55.18) runs Docker and Baget as a private NuGet server; the other (IP 10.129.55.110) hosts the CI/CD pipeline, the Kubernetes cluster, Harbor (or Tencent Cloud registry), Helm, and Jenkins. A third machine acts as a private Git repository.

CI/CD Flow

Git checkout → SonarQube static analysis → Docker build (pulling NuGet packages from Baget) → Tag and push the image to a private registry → Helm upgrade deploys the image to the Kubernetes cluster. Administrators can access the cluster via an Nginx reverse proxy for security.

Jenkins Installation

# Run Jenkins in Docker
docker run \
  -u root \
  -d \
  -p 8080:8080 \
  -p 50000:50000 \
  -v $(which docker):/bin/docker \
  -v /var/jenkins_home:/var/jenkins_home \
  -v /var/run/docker.sock:/var/run/docker.sock \
  --restart=always \
  --name=jenkinsci \
  jenkinsci/blueocean

After the container starts, edit /var/jenkins_home/updates/default.json to replace Google URLs with domestic mirrors:

cd /var/jenkins_home/updates
sed -i 's|http://updates.jenkins-ci.org/download|https://mirrors.tuna.tsinghua.edu.cn/jenkins|g' default.json
sed -i 's|http://www.google.com|https://www.baidu.com|g' default.json

Retrieve the initial admin password with:

cat /var/jenkins_home/secrets/initialAdminPassword

Helm Installation

# Download and install Helm 3.2.1
wget https://get.helm.sh/helm-v3.2.1-linux-amd64.tar.gz
tar -zxvf helm-v3.2.1-linux-amd64.tar.gz
cp linux-amd64/helm /usr/local/bin/

Baget (Private NuGet) Installation

Baget is deployed in Docker and protected by Nginx basic authentication because Baget does not yet support built‑in user permissions.

Harbor Installation

Follow the official Docker‑based Harbor guide; the image is pulled and run as a private container registry.

SonarQube Installation (Optional)

# PostgreSQL for SonarQube
docker run -d -p 6000:5432 -v /var/sonar/db:/var/lib/postgresql/data \
  -e POSTGRES_USER=sonar -e POSTGRES_PASSWORD=sonar \
  --name=sonar-postgres --restart=unless-stopped postgres:latest

# SonarQube server
docker run -d --name sonarqube \
  -p 9000:9000 \
  -e sonar.jdbc.url="jdbc:postgresql://192.888.10.122:6000/sonar" \
  -e sonar.jdbc.username=sonar \
  -e sonar.jdbc.password=sonar \
  -v sonarqube_conf:/opt/sonarqube/conf \
  -v sonarqube_extensions:/opt/sonarqube/extensions \
  -v sonarqube_logs:/opt/sonarqube/logs \
  -v sonarqube_data:/opt/sonarqube/data \
  --restart unless-stopped sonarqube

Dockerfile Example

FROM mcr.microsoft.com/dotnet/core/aspnet:3.1 AS base
WORKDIR /app
EXPOSE 80 443

FROM ccr.ccs.tencentyun.com/demaxiya/dotnet-core-sdk:3.1-buster-auth-nuget AS build
WORKDIR /src
COPY . .
RUN dotnet restore "Gy.HelloWorld.Api.csproj" -s http://nuget.demaxiya.com/v3/index.json -s https://api.nuget.org/v3/index.json
RUN dotnet build "Gy.HelloWorld.Api.csproj" -c Release -o /app/build

FROM build AS publish
RUN dotnet publish "Gy.HelloWorld.Api.csproj" -c Release -o /app/publish

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet","Gy.HelloWorld.Api.dll"]

Helm Chart Structure

A typical chart contains Chart.yaml, values.yaml, and template files such as deployment.yaml, service.yaml, and _helpers.tpl. Deploy with:

helm upgrade -i --namespace=${kube_namespace} \
  --set image.repository=${DOCKER_IMAGE} \
  --set image.tag=${GIT_TAG} \
  --set replicaCount=${params.replica_count} \
  --set nameOverride=${GIT_REPO} \
  ${GIT_REPO} ./deploy/helm/

Jenkinsfile Overview

def createTag() {
    def yesterday = sh(returnStdout:true, script:'date +"%Y-%m-%d" -d "-1 days"').trim()
    def commitCount = sh(returnStdout:true, script:"git rev-list HEAD --count --since=${yesterday}").trim()
    def dateNow = sh(returnStdout:true, script:'date "+%-m%d"').trim()
    return "v1.0.${dateNow}.${commitCount}"
}

pipeline {
    agent any
    environment {
        GIT_BRANCH = "${env.gitTargetBranch}"
        GIT_TAG = createTag()
        GIT_REPO = "${env.GIT_REPO}"
        DOCKER_CONTEXT_PATH = "${env.CONTEXT_PATH}"
        DOCKER_REGISTER_CREDS = credentials('e2e1cb5f-5538-43d7-2-14561')
        DOCKER_REGISTRY = "fwafa.421.42.com"
        DOCKER_NAMESPACE = "321"
        DOCKER_IMAGE = "${DOCKER_REGISTRY}/${DOCKER_NAMESPACE}/${GIT_REPO}"
        KUBECONFIG = "/root/.kube/config:/root/.kube/develop-config"
    }
    parameters { string(name:'replica_count', defaultValue:'2', description:'Container replica count') }
    stages {
        stage('Test Pipeline Stages') {
            when { environment name:'deploy_env', value:'test' }
            stages {
                stage('Params Analyze') { steps { echo "1. 参数检查" } }
                stage('Code Pull') {
                    steps {
                        echo "2. 代码拉取"
                        checkout([$class:'GitSCM', branches:[[name:'$gitTargetBranch']], userRemoteConfigs:[[credentialsId:'09d880cf-48af-4ce1-ac85-5f0c223e2559', url:"${env.GIT_URL}"]]])
                    }
                }
                stage('Code Analyze') { steps { echo "3. 代码静态检查" } }
                stage('Docker Build') {
                    steps {
                        echo "4. 构建Docker镜像"
                        sh "docker login -u ${DOCKER_REGISTER_CREDS_USR} -p ${DOCKER_REGISTER_CREDS_PSW} ${DOCKER_REGISTRY}"
                        sh "docker build --add-host nuget.degea31.com:182.132.313.37 -t ${DOCKER_IMAGE}:${GIT_TAG} ${DOCKER_CONTEXT_PATH}"
                        sh "docker push ${DOCKER_IMAGE}:${GIT_TAG}"
                        sh "docker rmi -f ${DOCKER_IMAGE}:${GIT_TAG}"
                    }
                }
            }
        }
        stage('Helm Deploy') {
            agent { docker { image 'wzrdtales/helm-kubectl' args '-u root:root -v /root/.kube:/root/.kube' } }
            steps {
                echo "5. 部署到K8s"
                script {
                    def kube_context = env.deploy_env == 'test' ? 'kubernetes-admin@kubernetes' : (env.deploy_env == 'pre-release' ? 'master' : 'kubernetes-admin@kubernetes')
                    def kube_namespace = env.deploy_env == 'test' ? 'test' : (env.deploy_env == 'pre-release' ? 'kube-pre-release' : '')
                    sh "kubectl config use-context ${kube_context}"
                    sh "cat ./deploy/helm/values.yaml > tmp.yaml"
                    sh "envsubst < tmp.yaml > ./deploy/helm/values.yaml"
                    sh "helm upgrade -i --namespace=${kube_namespace} --set image.repository=${DOCKER_IMAGE} --set image.tag=${GIT_TAG} --set replicaCount=${params.replica_count} --set nameOverride=${GIT_REPO} ${GIT_REPO} ./deploy/helm/"
                }
            }
        }
    }
}

Cleanup Script

# Delete old Jenkins builds
def jobName = "hello"
def maxNumber = 1888
Jenkins.instance.getItemByFullName(jobName).builds.findAll { it.number <= maxNumber }.each { it.delete() }