Using Jenkins Pipeline to Read YAML Configuration for Flexible CI/CD

With the rise of DevOps culture, Continuous Integration and Continuous Delivery (CI/CD) have become essential components of modern software development. Jenkins, an open‑source automation server, is widely used for building, testing, and deploying applications. This article discusses how to use a Jenkins Pipeline to read a YAML configuration file for a flexible CI/CD workflow, covering YAML authoring, storing it in a GitLab repository, and having the Jenkins Pipeline read and execute the configuration.

1. Writing the YAML File

YAML (YAML Ain't Markup Language) is a human‑readable data serialization format that is ideal for configuration files. Below is an example YAML file containing CI/CD configuration information:

ci_cd_config:
base_image: "maven:3.8.1-jdk-11"
maven_commands:
  build: "mvn clean package"
  test: "mvn test"
unit_test_enabled: true
sonarqube:
  enabled: true
  project_key: "my_project_key"
  project_name: "My Project"
  project_version: "1.0.0"

In this example we define the base Docker image, Maven build commands, a unit‑test toggle, and SonarQube code‑quality settings.

2. Storing the YAML in a GitLab Repository

Create a new GitLab repository (or use an existing one) and add the above YAML file (e.g., ci_cd_config.yaml ) to the project root. Follow these steps:

Create the GitLab repository: Log in to GitLab and click “New Project”. Enter a project name and description, then click “Create project”.

Upload the YAML file: Navigate to “Repository” → “Files”. Click “Upload file”, select the local ci_cd_config.yaml , and commit the changes.

2.1 Configure a GitLab Webhook to Trigger a Jenkins Job

To automatically trigger a Jenkins job after code is pushed, configure a webhook in GitLab:

Enter project settings: Go to “Settings” → “Webhooks”.

Add the webhook: In the “URL” field enter the Jenkins webhook address, e.g., http://your-jenkins-url/generic-webhook-trigger/invoke . Choose “Push events” as the trigger.

Save the webhook: Click “Add webhook”.

2.2 Configure Jenkins Generic Webhook Trigger

In Jenkins, install the “Generic Webhook Trigger” plugin and set up a job to respond to the GitLab webhook:

Install the plugin: Go to “Manage Jenkins” → “Manage Plugins”, search for and install “Generic Webhook Trigger”.

Create a Jenkins job: Create a new Pipeline job or use an existing one.

Configure the webhook trigger: In the job configuration, enable “Build Triggers” → “Generic Webhook Trigger” and define the conditions and parameters needed for the build.

Save the configuration: Ensure all changes are saved.

3. Jenkins Pipeline Reads the YAML Configuration

Next, configure the Jenkins Pipeline to read the YAML file and execute the CI/CD steps accordingly. Below is a sample Jenkinsfile :

pipeline {
    agent {
        docker {
            image "maven:3.8.1-jdk-11" // Fixed base image
        }
    }
    stages {
        stage('Checkout') {
            steps {
                git url: '[email protected]:your_username/your_repo.git', branch: 'main'
            }
        }
        stage('Read YAML') {
            steps {
                script {
                    def configFile = readYaml file: 'ci_cd_config.yaml'
                    env.MAVEN_BUILD_CMD = configFile.ci_cd_config.maven_commands.build
                    env.MAVEN_TEST_CMD = configFile.ci_cd_config.maven_commands.test
                    env.UNIT_TEST_ENABLED = configFile.ci_cd_config.unit_test_enabled.toString()
                    env.SONARQUBE_ENABLED = configFile.ci_cd_config.sonarqube.enabled.toString()
                }
            }
        }
        stage('Build') {
            steps {
                sh "${MAVEN_BUILD_CMD}"
            }
        }
        stage('Test') {
            when { expression { env.UNIT_TEST_ENABLED == 'true' } }
            steps {
                sh "${MAVEN_TEST_CMD}"
            }
        }
        stage('SonarQube Analysis') {
            when { expression { env.SONARQUBE_ENABLED == 'true' } }
            steps {
                sh "mvn sonar:sonar -Dsonar.projectKey=${configFile.ci_cd_config.sonarqube.project_key} -Dsonar.projectName=${configFile.ci_cd_config.sonarqube.project_name} -Dsonar.projectVersion=${configFile.ci_cd_config.sonarqube.project_version}"
            }
        }
    }
}

3.1 Key Steps Explained

Read YAML file: Use readYaml to load ci_cd_config.yaml into the configFile variable.

Set environment variables: Extract values from the YAML into environment variables for later stages.

Conditional execution: Use the when directive to run unit tests and SonarQube analysis only when they are enabled in the configuration.

Conclusion

By storing CI/CD configuration in a YAML file within GitLab and having a Jenkins Pipeline read that configuration, teams can achieve a flexible and scalable continuous integration and delivery process. The webhook setup ensures that GitLab commits automatically trigger Jenkins builds, improving readability, maintainability, and overall workflow management. Future work may focus on further optimizing Jenkins Pipeline performance and scalability for more complex scenarios.

📚 Free Book Giveaway: Win "DevOps: Enterprise‑Level CI/CD Practice"!

In today’s fast‑moving tech era, DevOps is key to improving software delivery speed and quality. To help enthusiasts and professionals deepen their understanding of DevOps, we are offering a free book promotion.

Book Overview: "DevOps: Enterprise‑Level CI/CD Practice" is a practical guide that delves into best practices for continuous integration and delivery, featuring real‑world case studies and actionable solutions for implementing CI/CD in enterprise environments.

How to Participate:

Follow our social media account (DevOps Cloud Classroom).

Leave a comment on this article sharing your enterprise‑level Jenkins experience.

The activity ends on March 29, 2025 12:00:00 . The top 5 commenters by likes will each receive a copy of the book.

Why read this book?

Highly practical: Real‑case solutions you can apply immediately.

Easy to understand: Clear language suitable for readers of all levels.

Latest technologies: Covers popular DevOps tools such as Jenkins, Docker, Kubernetes, and more.

Join now! Let this book help you set sail on your DevOps journey, boost your technical skills, and enhance your career competitiveness. We look forward to your participation—good luck! 🚀

Note: The organizer reserves the final right of interpretation for the activity.