How to Transform Jenkins Master‑Slave CI into a Scalable Kubernetes‑Based Architecture

Background

The existing Jenkins CI infrastructure runs on a static master‑slave architecture using physical machines, primarily for unit and integration tests.

Problems with the Traditional Setup

Adding new slaves requires manual provisioning and software installation.

Resource allocation is inefficient: some slaves are idle while others queue jobs.

Each slave has slight configuration differences, increasing maintenance overhead.

If the master fails, the entire pipeline becomes unavailable.

Kubernetes‑Based Solution

To address these issues, the team migrated Jenkins to Kubernetes, allowing the master and slaves to run as pods that are dynamically scheduled. The master runs on a single node, while slaves are created on demand based on job labels and automatically removed after the job finishes, ensuring optimal resource utilization.

Deploying Jenkins Master

A custom Jenkins master Docker image is built (or the official jenkins/jenkins:lts can be used) with two exposed ports: the web UI port and the JNLP agent port. The JNLP port must be reachable from outside the container so that dynamically created slave pods can register with the master.

Deploying SonarQube

SonarQube is packaged together with a MySQL database in a single Docker image. The MySQL image includes initialization scripts ( privileges.sql, sonar.sql, init_sonar.sql) and a startup script ( start.sh) that creates the database, loads initial data, and starts MySQL.

FROM mysql:5.7
ENV MYSQL_ALLOW_EMPTY_PASSWORD yes
# Copy configuration and initialization files
COPY mysqld.cnf /etc/mysql/mysql.conf.d/
COPY setup.sh /mysql/setup.sh
COPY privileges.sql /mysql/privileges.sql
COPY sonar.sql /mysql/sonar.sql
COPY init_sonar.sql /mysql/init_sonar.sql
COPY run-entrypoint.sh /mysql/run-entrypoint.sh
COPY start.sh /mysql/start.sh
ENTRYPOINT ["/mysql/run-entrypoint.sh","/mysql/setup.sh"]

Building and Publishing Images

Images are built, tagged, and pushed to a private Harbor registry:

docker build -t [IMAGE:TAG] .
 docker tag SOURCE_IMAGE harbor.xxx.com/xxx/IMAGE[:TAG]
 docker push harbor.xxx.com/xxx/IMAGE[:TAG]

Kubernetes Resources

Key resources include Deployment, Service, and Ingress:

Deployment declares the desired state for Jenkins and SonarQube pods, enabling declarative updates.

Service uses label selectors to provide a stable endpoint for the pods, abstracting away pod scaling and restarts.

Ingress consolidates external access, reducing the need for multiple NodePort or LoadBalancer services. Annotations such as nginx.ingress.kubernetes.io/proxy-body-size: 600m are required for large file uploads.

Example CORS configuration for the Ingress:

nginx.ingress.kubernetes.io/allow-headers: DNT,X-CustomHeader,Keep-Alive,User-Agent,Requested-With,If-Modified-Since,Cache-Control,Content-Type,Authorization
nginx.ingress.kubernetes.io/allow-methods: PUT,GET,POST,OPTIONS
nginx.ingress.kubernetes.io/allow-origin: "*"
nginx.ingress.kubernetes.io/enable-cors: "true"
nginx.ingress.kubernetes.io/proxy-body-size: 600m

Web Terminal Integration

A web‑based terminal built with xterm.js connects to a backend k8s‑websocket service. The service uses the Kubernetes remotecommand package to open an SSH‑like channel to the container. Terminal resize events are forwarded by setting the resize channel byte to 4 in remotecommand.go.

Tuning Jenkins Provisioning

By default Jenkins throttles the creation of new agents. To force immediate provisioning for each queued build, adjust the following system properties when starting the master:

Dhudson.slaves.NodeProvisioner.initialDelay=0

Dhudson.slaves.NodeProvisioner.MARGIN=50

Dhudson.slaves.NodeProvisioner.MARGIN0=0.85

Conclusion

Moving Jenkins CI/CD to Kubernetes provides high availability, dynamic scaling, and better resource efficiency, while the integrated SonarQube image and web terminal further streamline development workflows. However, mastering Kubernetes concepts and fine‑tuning Jenkins parameters remain essential for a smooth migration.