Serverless Jenkins and Prow: Cloud‑Native CI/CD on Kubernetes

Jenkins, originally created as Hudson in 2004, has become a leading CI/CD tool with millions of installations, but its traditional server model presents challenges such as single‑point failures, disk‑space exhaustion, plugin version conflicts, GitHub rate limits, and high JVM memory usage.

To address these issues, Jenkins X promotes a cloud‑native evolution, enabling Jenkins to run only when needed, using temporary pipeline engines, providing high‑availability webhook handling, and storing job configurations in Git for disaster recovery.

The project integrates Prow, an event‑driven, highly available webhook processor from the Kubernetes ecosystem, which writes ProwJob CRDs to Kubernetes and supports automatic PR merging based on configurable rules.

Knative Build extends Kubernetes with CRDs to define build templates that run container steps sequentially, allowing flexible, container‑based builds within the cluster.

Serverless Jenkins combines these technologies, automatically installing and configuring Prow and Knative on Kubernetes, generating necessary configurations via the jx CLI, and executing builds in temporary Jenkins instances that are discarded after completion, dramatically reducing startup time.

To try it out, you can provision the environment with Terraform and create a GKE cluster using the following commands:

jx create terraform
jx create cluster gke --prow
jx install --prow

Frequently asked questions cover the lack of a traditional Jenkins UI (replaced by Prow’s Deck UI), where to find build logs (via JenkinsfileRunner output to standard output and jx logs -k ), and compatibility of environment variables in Jenkinsfiles.

Migrating existing Jenkinsfiles to Serverless Jenkins involves changing the agent to "any", removing container blocks, and adjusting checkout steps to accommodate the new architecture.

Current limitations include GitHub‑only support, pending upstream migration of Jenkins X forks, untested scripted pipelines, and lack of support for Kubernetes plugin PodTemplates.

In summary, Jenkins X offers a comprehensive, cloud‑native CI/CD platform that leverages Prow, Knative, and Serverless Jenkins to provide automated, scalable, and cost‑effective build and deployment pipelines on Kubernetes.