Configuring Jenkins for Same‑Cluster and Cross‑Cluster Kubernetes CI/CD Integration

1. Overview

This article shares a concrete implementation of CI/CD integration between a k8s environment and jenkins , focusing on the challenges of connecting Jenkins to Kubernetes clusters in different environments.

Problem: Different environments require distinct Jenkins‑to‑Kubernetes connection configurations.

Reasons for multiple environments include:

Legacy Jenkins and dedicated slave nodes that cannot be fully cloud‑native.

Multiple clusters sharing a single Jenkins server, e.g., a central infrastructure cluster A and business clusters B , C , D .

We define two scenarios:

Same‑cluster: Jenkins runs inside a Kubernetes cluster and connects to that same cluster.

Cross‑cluster: Jenkins connects to an external Kubernetes cluster, or connects to another cluster from a different environment.

2. Same‑Cluster Configuration

When Jenkins is deployed inside the target k8s cluster, the connection is native. After installing the Kubernetes plugin in Jenkins, configure the connection address directly.

Required RBAC objects (ServiceAccount, Role, RoleBinding) are applied inside the kube-system namespace:

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins
  namespace: kube-system
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: jenkins
  namespace: kube-system
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/exec"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/log"]
  verbs: ["get","list","watch"]
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: jenkins
  namespace: kube-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: jenkins
subjects:
- kind: ServiceAccount
  name: jenkins
  namespace: kube-system

After creating these resources, set the Kubernetes cloud URL in Jenkins System Configuration → Kubernetes → Kubernetes URL.

3. Cross‑Cluster Configuration

Scenario: Jenkins runs in cluster A (or on a VM) and needs to launch pods in external cluster B .

3.1 Required Ports

HTTP port (default 8080, or 80/443 if behind a reverse proxy).

Agent port (JNLP) – TCP 50000.

SSH port – rarely used for Jenkins agents.

3.2 Network Policy

Open the following ports between clusters:

Cluster B nodes must reach Jenkins’s HTTP and Agent ports.

Cluster A (Jenkins server) must reach the API server of cluster B.

If the Jenkins UI is protected by an Ingress whitelist, add the source IPs of cluster B to the whitelist.

3.3 Certificate Generation and Configuration

3.3.1 kubeconfig Structure

A typical kubeconfig contains three certificate sections: ca.crt , client.crt , and client.key .

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: xxx
    server: https://
:6443
  name: cluster1
contexts:
- context:
    cluster: cluster1
    user: admin
  name: context-cluster1-admin
current-context: context-cluster1-admin
kind: Config
preferences: {}
users:
- name: admin
  user:
    client-certificate-data: xxx
    client-key-data: xxx

3.3.2 Convert to PKCS#12

Extract certificates using yq and base64 :

yq e '.clusters[0].cluster.certificate-authority-data' .kube/config | base64 -d > ca.crt

yq e '.users[0].user.client-certificate-data' .kube/config | base64 -d > client.crt

yq e '.users[0].user.client-key-data' .kube/config | base64 -d > client.key

Then create a PKCS#12 file:

openssl pkcs12 -export -out cert.pfx -inkey client.key -in client.crt -certfile ca.crt
Enter Export Password:  # set a password
Verifying - Enter Export Password:

3.3.3 Import into Jenkins

In Jenkins → Manage Credentials, add a global credential of type Certificate , choose “Upload PKCS#12 certificate”, and upload cert.pfx with the password set above.

3.4 Configure External Kubernetes Cloud in Jenkins

Navigate to Jenkins → Manage → Configure System → Add a new Kubernetes cloud.

Name: Alias for the external cluster.

Kubernetes URL: https:// :6443 (the API server address of the external cluster).

Server Certificate Key: Base64‑decoded content of the key from the PKCS#12 file.

Namespace: Namespace where Jenkins agents will be created (e.g., jenkins ).

Credentials: Select the imported PKCS#12 certificate.

Jenkins URL: URL of the Jenkins UI in cluster A.

Click “Test Connection” to verify the cross‑cluster link.

4. Testing and Validation

4.1 Configure Pod Template

Define a pod template that will be used by Jenkins agents in the external cluster. This template can be referenced by freestyle or pipeline jobs.

4.2 Freestyle Job Test

Create a freestyle job, restrict it to the label of the pod template (e.g., k8s-test-cluster ), and run a simple shell command to verify the agent runs in the external cluster.

4.3 Pipeline Job Test

Example pipeline that uses the same pod label:

pipeline{
    agent{ node{ label 'k8s-test-cluster-jnlp-slave' } }
    stages{
        stage('Deploy to Kubernetes'){
            steps{
                script{
                    sh """
                    kubectl version
                    kubectl get cs
                    """
                }
            }
        }
    }
}

After building, check the console output to confirm successful interaction with the external Kubernetes cluster.

With these steps, Jenkins can reliably connect to both same‑cluster and cross‑cluster Kubernetes environments for CI/CD workloads.

See you ~