Mastering Jenkins Pipelines: A Step‑by‑Step Guide for Beginners

Jenkins Pipeline Overview

Jenkins supports two pipeline syntaxes: Declarative Pipeline (recommended for new versions) and Scripted Pipeline (used in older Jenkins versions). Both define the build process inside a pipeline{} block.

Declarative Pipeline

In a Declarative pipeline the entire workflow is described inside a pipeline{} block. The top‑level agent defines where the pipeline runs, and stages contain individual stage blocks such as Build, Test, and Deploy. Each stage contains a steps section that runs the actual commands.

// Jenkinsfile (Declarative Pipeline)
pipeline {
  agent any
  stages {
    stage('Build') {
      steps { echo 'Build' }
    }
    stage('Test') {
      steps { echo 'Test' }
    }
    stage('Deploy') {
      steps { echo 'Deploy' }
    }
  }
}

Key Directives

agent : defines the execution node. Options include any, none, label 'my-label', node { … }, dockerfile { … }, docker { … }, and kubernetes { … }.

stage : a logical phase of the pipeline; the name is user‑defined.

steps : the actual commands executed in a stage (e.g., sh, echo).

Agent Variants

agent any

– run on any available agent.

pipeline { agent any }

agent none

– no global agent; each stage must define its own agent.

pipeline {
  agent none
  stages {
    stage('Build') { agent any }
  }
}

agent { label 'my-defined-label' }

– select a node by label.

pipeline {
  agent { label 'my-defined-label' }
}

node

with custom workspace.

pipeline {
  agent none
  stages {
    stage('Build') {
      agent {
        node {
          label 'role-master'
          customWorkspace "/tmp/zhangzhuo/data"
        }
      }
      steps { sh "echo role-master > 1.txt" }
    }
  }
}

dockerfile

– build a container from a Dockerfile in the source tree.

pipeline {
  agent {
    dockerfile {
      filename 'Dockerfile.build'
      dir 'build'
      label 'role-master'
      additionalBuildArgs '--build-arg version=1.0.2'
    }
  }
}

docker

– run the pipeline inside an existing Docker image.

pipeline {
  agent {
    docker {
      image '192.168.10.15/kubernetes/alpine:latest'
      label 'role-master'
      args '-v /tmp:/tmp'
    }
  }
}

kubernetes

– run on a Kubernetes pod with a custom YAML definition and workspace volume.

pipeline {
  agent {
    kubernetes {
      cloud 'kubernetes'
      slaveConnectTimeout 1200
      workspaceVolume emptyDirWorkspaceVolume()
      yaml '''
kind: Pod
metadata:
  name: jenkins-agent
spec:
  containers:
  - args: ['$(JENKINS_SECRET)','$(JENKINS_NAME)']
    image: '192.168.10.15/kubernetes/jnlp:alpine'
    name: jnlp
    imagePullPolicy: IfNotPresent
  - command: ['cat']
    image: '192.168.10.15/kubernetes/alpine:latest'
    name: kubectl
    tty: true
''' 
    }
  }
}

Scripted Pipeline

A Scripted pipeline uses raw Groovy code. It typically contains one or more node blocks that execute the core steps.

// Jenkinsfile (Scripted Pipeline)
node {
  stage('Build') { echo 'Build' }
  stage('Test')  { echo 'Test' }
  stage('Deploy'){ echo 'Deploy' }
}

Post‑Build Actions

The post section runs after a pipeline or stage finishes. Supported conditions are always, changed, fixed, regression, failure, success, unstable, aborted, unsuccessful, and cleanup.

// Jenkinsfile (Declarative Pipeline)
pipeline {
  agent any
  stages {
    stage('Example1') { steps { echo 'Hello World1' } }
    stage('Example2') { steps { echo 'Hello World2' } }
  }
  post {
    always { echo 'I will always say Hello again!' }
  }
}

Steps, Directives and Sections

steps execute commands within a stage, e.g., sh "env" or multi‑line shell scripts. Directives include environment, options, parameters, triggers, input, when, and parallel. Each is described below.

Environment

Defines environment variables for the whole pipeline or a specific stage. Credentials can be injected via credentials(). Example:

pipeline {
  agent any
  environment {
    NAME = 'zhangzhuo'
  }
  stages {
    stage('env1') {
      environment { HARBOR = 'https://192.168.10.15' }
      steps { sh "env" }
    }
    stage('env2') { steps { sh "env" } }
  }
}

When using a username/password credential (e.g., harbor-account) Jenkins creates three variables: HARBOR ("user:pass"), HARBOR_USR, and HARBOR_PSW.

Options

Built‑in directives that affect pipeline execution, such as: buildDiscarder: keep a limited number of builds. disableConcurrentBuilds: prevent parallel runs. disableResume: stop automatic resume after controller restart. newContainerPerStage: create a fresh container for each stage when using Docker. quietPeriod: wait before starting the pipeline. retry: retry failed steps. timeout: abort after a time limit. timestamps: add timestamps to console output.

Options can be placed at the pipeline level or inside a stage (only retry, timeout, timestamps and stage‑specific options are allowed inside a stage).

pipeline {
  agent any
  options {
    timeout(time: 1, unit: 'HOURS')
    timestamps()
    buildDiscarder(logRotator(numToKeepStr: '3'))
    quietPeriod(10)
    retry(3)
  }
  stages {
    stage('env1') { steps { sh "env"; sleep 2 } }
    stage('env2') { steps { sh "env" } }
  }
}

Parameters

Define user‑supplied values when manually triggering a pipeline. Supported types are string, text, booleanParam, choice, and password. Example:

pipeline {
  agent any
  parameters {
    string(name: 'DEPLOY_ENV', defaultValue: 'staging')
    text(name: 'DEPLOY_TEXT', defaultValue: 'One
Two
Three')
    booleanParam(name: 'DEBUG_BUILD', defaultValue: true)
    choice(name: 'CHOICES', choices: ['one','two','three'])
    password(name: 'PASSWORD', defaultValue: 'SECRET')
  }
  stages { /* ... */ }
}

Plugin‑provided parameter types include imageTag (requires Image Tag Parameter plugin) and gitParameter (requires Git Parameter plugin).

Triggers

Automate pipeline starts via cron, upstream, pollSCM, or webhooks. Cron syntax uses H (hash) to spread load.

pipeline {
  agent any
  triggers {
    cron('H */4 * * 1-5')   // every 4 hours on weekdays
    cron('H/12 * * * *')    // every 12 minutes
    cron('H * * * *')       // every hour
  }
  stages { /* ... */ }
}

Upstream trigger runs the pipeline only when another job finishes with a specific result (e.g., SUCCESS).

pipeline {
  agent any
  triggers { upstream(upstreamProjects: 'env', threshold: hudson.model.Result.SUCCESS) }
  stages { /* ... */ }
}

Input

Interactive prompts that pause the pipeline for user confirmation. Parameters can be collected, and submission can be limited to specific users.

pipeline {
  agent any
  stages {
    stage('Example') {
      input {
        message "Continue?"
        ok "Proceed"
        submitter "alice,bob"
        parameters { string(name: 'PERSON', defaultValue: 'Mr Jenkins') }
      }
      steps { echo "Hello, ${PERSON}" }
    }
  }
}

When

Conditional execution of a stage based on criteria such as branch name, environment variable, expression, or combination operators ( not, allOf, anyOf).

pipeline {
  agent any
  stages {
    stage('Deploy') {
      when {
        allOf {
          branch 'master'
          environment name: 'DEPLOY_TO', value: 'production'
        }
      }
      steps { echo 'Deploying' }
    }
  }
}

Supported conditions include branch, changelog, environment, equals, expression, tag, not, allOf, and anyOf. Pre‑stage evaluation can be controlled with beforeAgent, beforeInput, and beforeOptions (priority: beforeOptions > beforeInput > beforeAgent).

Parallel

Run multiple branches concurrently. failFast true aborts remaining branches if any fails.

pipeline {
  agent any
  stages {
    stage('Parallel Stage') {
      failFast true
      parallel {
        stage('Branch A') { steps { echo "On Branch A" } }
        stage('Branch B') { steps { echo "On Branch B" } }
        stage('Branch C') {
          stages {
            stage('Nested 1') { steps { echo "In Nested 1" } }
            stage('Nested 2') { steps { echo "In Nested 2" } }
          }
        }
      }
    }
  }
}

Using a Jenkinsfile

Storing the pipeline definition in a Jenkinsfile inside source control enables versioning, code review, and collaboration.

Built‑in Environment Variables

Common variables include BUILD_ID, BUILD_NUMBER, BUILD_TAG, BUILD_URL, JOB_NAME, NODE_NAME, JENKINS_URL, and WORKSPACE. They can be accessed via env.VAR_NAME.

pipeline {
  agent any
  stages {
    stage('Print Env') {
      steps {
        echo "BUILD_ID: $env.BUILD_ID"
        echo "BUILD_NUMBER: $env.BUILD_NUMBER"
        echo "BUILD_TAG: $env.BUILD_TAG"
        echo "BUILD_URL: $env.BUILD_URL"
        echo "JOB_NAME: $env.JOB_NAME"
        echo "NODE_NAME: $env.NODE_NAME"
        echo "JENKINS_URL: $env.JENKINS_URL"
        echo "WORKSPACE: $env.WORKSPACE"
      }
    }
  }
}

Dynamic Variables

Assign values based on command output using returnStdout or returnStatus:

pipeline {
  agent any
  environment {
    CC = "${sh(returnStdout: true, script: 'echo -n "clang"')}"
    EXIT_STATUS = "${sh(returnStatus: true, script: 'exit 1')}"
  }
  stages {
    stage('Show') { steps { sh "printenv" } }
  }
}

Credentials Management

Use credentials() to inject secret text, username/password, or secret files into the environment.

Secret Text

pipeline {
  agent any
  environment {
    AWS_ACCESS_KEY_ID = credentials('txt1')
    AWS_SECRET_ACCESS_KEY = credentials('txt2')
  }
  stages {
    stage('Show') { steps { echo "$AWS_ACCESS_KEY_ID" } }
  }
}

Username/Password – creates three variables: VAR ("user:pass"), VAR_USR, and VAR_PSW.

pipeline {
  agent any
  environment {
    BITBUCKET_COMMON_CREDS = credentials('harbor-account')
  }
  stages {
    stage('Print') { steps { sh "env" } }
  }
}

Secret File (e.g., kubeconfig)

pipeline {
  agent {
    kubernetes {
      cloud 'kubernetes'
      slaveConnectTimeout 1200
      workspaceVolume emptyDirWorkspaceVolume()
      yaml '''
kind: Pod
metadata:
  name: jenkins-agent
spec:
  containers:
  - args: ['$(JENKINS_SECRET)','$(JENKINS_NAME)']
    image: '192.168.10.15/kubernetes/jnlp:alpine'
    name: jnlp
    imagePullPolicy: IfNotPresent
  - command: ['cat']
    image: '192.168.10.15/kubernetes/kubectl:apline'
    name: kubectl
    tty: true
''' 
    }
  }
  environment { MY_KUBECONFIG = credentials('kubernetes-cluster') }
  stages {
    stage('kubectl') {
      steps {
        container(name: 'kubectl') {
          sh "kubectl get pod -A --kubeconfig $MY_KUBECONFIG"
        }
      }
    }
  }
}

These examples demonstrate how to configure agents, define stages, use environment variables, apply options, accept parameters, trigger builds, request input, conditionally execute stages, run parallel branches, and manage credentials within Declarative and Scripted Jenkins pipelines.