How to Automate Jenkins, Gogs, and Nexus Deployment on Kubernetes with a Custom Template Engine

This article details the automation of CI/CD components (Jenkins, Gogs, Nexus) on a shared Kubernetes cluster used for a workshop. It explains how a lightweight template engine, Kubernetes manifests, initContainers, and one‑off Jobs are combined to achieve fully reproducible provisioning.

Variable Template Engine

Each workshop participant receives isolated namespaces such as cicd-<suffix>, dev-<suffix>, and stage-<suffix>. The <suffix> value is stored during the initial CI/CD installation and later injected into Jenkins jobs so they know which namespace to target.

The engine is implemented in ./tmpl.sh. It reads a variable file and environment variables, substitutes placeholders marked with $$ in YAML files, and writes the rendered manifest to stdout. The output is piped to kubectl apply -f -. This approach works on kubelet versions that do not yet include the built‑in kustomize sub‑command.

Automated Jenkins Installation

The manifest cicd-infra/jenkins.yaml (~600 lines) defines the following key resources:

ServiceAccount jenkins – used by the Jenkins pod and its build‑agent pods.

RoleBinding jenkins_edit – grants the ServiceAccount the required permissions.

Service jenkins-jnlp – endpoint for the JNLP build‑agent connection.

Ingress jenkins-ingress – exposes the Jenkins UI.

ConfigMaps jenkins-jobs and jenkins – contain job definitions and the initial Jenkins configuration.

Deployment jenkins – runs two containers:

An installer initContainer that executes /var/jenkins_config/apply_config.sh (mounted from the ConfigMap). This script installs plugins listed in plugins.txt and creates built‑in jobs.

The main Jenkins container, which starts after the initContainer finishes and inherits the pre‑installed plugins and configuration via shared volumes.

Environment variables JENKINS_UC and JENKINS_UC_DOWNLOAD point to a domestic mirror to accelerate plugin downloads.

Key plugins (as of the workshop) include:

git

dashboard-view

pipeline-stage-view

workflow-aggregator

kubernetes:1.20.0

– enables Jenkins to launch Kubernetes pods as build agents for .NET and image‑builder workloads.

Two shared volumes ( jenkins-home and plugin-dir) allow the main Jenkins container to reuse the files written by the installer.

Automated Gogs and Nexus Installation

Manifests for Gogs and Nexus are simpler and use emptyDir volumes for temporary storage (no persistent storage is provisioned for the workshop). After the services become ready, a one‑time Job named cicd-installer performs additional initialization:

Wait for gogs-postgresql and gogs pods, then call Gogs’s RESTful API to create a user account, import micro‑service repositories from GitHub, and configure webhooks.

Wait for the nexus pod, then invoke Nexus’s Script API to change the default admin password to admin/admin.

The Gogs automation relies on standard HTTP calls, while Nexus uses its scripting API, which offers greater flexibility but also introduces additional security considerations.

Summary of Automation Techniques

Deploy applications with Deployment resources (Jenkins, Gogs, Nexus, etc.).

Use initContainer for pre‑deployment tasks such as plugin installation.

Share files between containers via shared volumes.

Inject configuration through environment variables (e.g., plugin download mirrors).

Mount static configuration files with ConfigMap resources.

Track rollout status with readiness/liveness probes and kubectl rollout status.

Execute one‑off tasks with a Job (e.g., cicd-installer).

Build custom container images using Dockerfile for specialized Jenkins agents.

Run built‑in scripts (e.g., install-plugins.sh) for automated configuration.

Interact with services via RESTful APIs (Gogs) or Script APIs (Nexus, Jenkins).

Replace variable placeholders with the custom template engine.

In production environments, persistent storage should be added by configuring appropriate StorageClass objects and enabling dynamic provisioning, which the workshop scripts intentionally omit for simplicity.

Original article: https://blog.jijiechen.com/post/cicd-infra-as-code-on-kubernetes/