Master Jenkins Pipelines: Declarative vs Scripted, Agents, Stages, and Advanced Features

1. What Is a Pipeline

Jenkins provides two pipeline types: Declarative (recommended for new versions) and Scripted (used in older Jenkins versions).

1.1 Declarative Pipeline

In Declarative syntax, the pipeline is defined inside a pipeline{} block, which contains all work to be performed.

Parameter Description:

agent any: Execute the pipeline on any available agent or a specific node.

stage: Define a pipeline stage (e.g., Build, Test, Deploy). The name is user‑defined.

steps: Execute the concrete steps for a stage.

// Jenkinsfile (Declarative Pipeline)
pipeline {
  agent any
  stages {
    stage('Build') {
      steps {
        echo 'Build'
      }
    }
    stage('Test') {
      steps {
        echo 'Test'
      }
    }
    stage('Deploy') {
      steps {
        echo 'Deploy'
      }
    }
  }
}

1.2 Scripted Pipeline

In Scripted syntax, one or more node blocks perform the core work.

Parameter Description:

node: Execute the pipeline on any available agent or a specific node.

stage: Same meaning as in Declarative; optional in Scripted pipelines but useful for UI visibility.

// Jenkinsfile (Scripted Pipeline)
node {
  stage('Build') {
    echo 'Build'
  }
  stage('Test') {
    echo 'Test'
  }
  stage('Deploy') {
    echo 'Deploy'
  }
}

2. Declarative Pipeline Details

A Declarative pipeline must be wrapped in a pipeline{} block.

pipeline {
  /* insert Declarative Pipeline here */
}

Key rules for Declarative syntax:

The top level must be a pipeline{} block.

Semicolons are optional; each statement must be on its own line.

Blocks can contain Sections, Directives, Steps, or assignment statements.

Property references are treated as parameterless method calls (e.g., input becomes input()).

2.1 Sections

Sections are not keywords; they are code regions that may contain Agent, Stages, post, Directives, and Steps.

1. Agent

The Agent defines where the pipeline or a specific stage runs. It can be defined globally or per stage.

any

Run on any available agent.

pipeline {
  agent any
}

none

No global agent; each stage must define its own agent.

pipeline {
  agent none
  stages {
    stage('Stage For Build') {
      agent any
    }
  }
}

label

Select a node by label, e.g., agent { label 'my-defined-label' }.

pipeline {
  agent none
  stages {
    stage('Stage For Build') {
      agent { label 'role-master' }
      steps { echo "role-master" }
    }
  }
}

node

Similar to label but allows extra configuration such as customWorkspace.

pipeline {
  agent none
  stages {
    stage('Stage For Build') {
      agent {
        node {
          label 'role-master'
          customWorkspace "/tmp/zhangzhuo/data"
        }
      }
      steps { sh "echo role-master > 1.txt" }
    }
  }
}

dockerfile

Build a container from a Dockerfile in the source repository.

agent {
  dockerfile {
    filename 'Dockerfile.build'
    dir 'build'
    label 'role-master'
    additionalBuildArgs '--build-arg version=1.0.2'
  }
}

docker

Use an external Docker image directly.

agent {
  docker {
    image '192.168.10.15/kubernetes/alpine:latest'
    label 'role-master'
    args '-v /tmp:/tmp'
  }
}

kubernetes

Requires the Kubernetes plugin. Example creates a dynamic slave pod.

agent {
  kubernetes {
    cloud 'kubernetes'
    slaveConnectTimeout 1200
    workspaceVolume emptyDirWorkspaceVolume()
    yaml '''
kind: Pod
metadata:
  name: jenkins-agent
spec:
  containers:
  - args: ['$(JENKINS_SECRET)', '$(JENKINS_NAME)']
    image: '192.168.10.15/kubernetes/jnlp:alpine'
    name: jnlp
    imagePullPolicy: IfNotPresent
  - command: ['cat']
    image: '192.168.10.15/kubernetes/alpine:latest'
    name: date
    tty: true
  restartPolicy: Never
''' 
  }
}

2. Agent Configuration Examples

Kubernetes Example

pipeline {
  agent {
    kubernetes {
      cloud 'kubernetes'
      slaveConnectTimeout 1200
      workspaceVolume emptyDirWorkspaceVolume()
      yaml '''
kind: Pod
metadata:
  name: jenkins-agent
spec:
  containers:
  - args: ['$(JENKINS_SECRET)', '$(JENKINS_NAME)']
    image: '192.168.10.15/kubernetes/jnlp:alpine'
    name: jnlp
    imagePullPolicy: IfNotPresent
  - command: ['cat']
    image: '192.168.10.15/kubernetes/alpine:latest'
    name: date
    tty: true
  - command: ['cat']
    image: '192.168.10.15/kubernetes/kubectl:apline'
    name: kubectl
    tty: true
  restartPolicy: Never
''' 
    }
  }
  environment { MY_KUBECONFIG = credentials('kubernetes-cluster') }
  stages {
    stage('Data') {
      steps { container(name: 'date') { sh "date" } }
    }
    stage('echo') {
      steps { container(name: 'date') { sh "echo 'k8s is pod'" } }
    }
    stage('kubectl') {
      steps { container(name: 'kubectl') { sh "kubectl get pod -A --kubeconfig $MY_KUBECONFIG" } }
    }
  }
}

Docker Example

pipeline {
  agent none
  stages {
    stage('Example Build') {
      agent { docker 'maven:3-alpine' }
      steps { echo 'Hello, Maven'; sh 'mvn --version' }
    }
    stage('Example Test') {
      agent { docker 'openjdk:8-jre' }
      steps { echo 'Hello, JDK'; sh 'java -version' }
    }
  }
}

3. Post

always – runs regardless of the stage result.

changed – runs only when the current result differs from the previous run.

fixed – runs when the current build succeeds after a previous failure.

regression – runs when the current build fails after a previous success.

failure – runs only on failure.

success – runs only on success.

unstable – runs on unstable builds.

aborted – runs when the build is manually aborted.

unsuccessful – runs when the result is not success.

cleanup – runs after all other post actions.

// Jenkinsfile (Declarative Pipeline)
pipeline {
  agent any
  stages {
    stage('Example1') { steps { echo 'Hello World1' } }
    stage('Example2') { steps { echo 'Hello World2' } }
  }
  post {
    always { echo 'I will always say Hello again!' }
  }
}

4. Steps

Steps execute commands within a stage, e.g., sh or echo.

// Jenkinsfile (Declarative Pipeline)
pipeline {
  agent any
  stages {
    stage('Example') {
      steps {
        echo 'Hello World'
        sh "echo 'Hello World1'
 echo 'Hello World2'"
      }
    }
  }
}

2.2 Directives

Directives provide additional configuration such as environment, options, parameters, triggers, input, and when.

1. Environment

Define environment variables globally or per stage. The credentials() method can inject secret text or username/password pairs.

// Example
pipeline {
  agent any
  environment { NAME = 'zhangzhuo' }
  stages {
    stage('env1') {
      environment { HARBOR = 'https://192.168.10.15' }
      steps { sh "env" }
    }
    stage('env2') { steps { sh "env" } }
  }
}

2. Options

Built‑in options like timeout , retry , timestamps , etc., control pipeline behavior.

pipeline {
  agent any
  options {
    timeout(time: 1, unit: 'HOURS')
    timestamps()
    buildDiscarder(logRotator(numToKeepStr: '3'))
    quietPeriod(10)
    retry(3)
  }
  stages { /* ... */ }
}

3. Parameters

Parameters let users supply values when triggering a pipeline (string, text, boolean, choice, password, etc.).

pipeline {
  agent any
  parameters {
    string(name: 'DEPLOY_ENV', defaultValue: 'staging', description: '1')
    text(name: 'DEPLOY_TEXT', defaultValue: 'One
Two
Three
', description: '2')
    booleanParam(name: 'DEBUG_BUILD', defaultValue: true, description: '3')
    choice(name: 'CHOICES', choices: ['one','two','three'], description: '4')
    password(name: 'PASSWORD', defaultValue: 'SECRET', description: 'A secret password')
    imageTag(name: 'DOCKER_IMAGE', image: 'kubernetes/kubectl', registry: 'https://192.168.10.15', credentialId: 'harbor-account')
    gitParameter(name: 'BRANCH', branchFilter: 'origin/(.*)', description: 'Branch for build and deploy')
  }
  stages { /* ... */ }
}

4. Triggers

Automatic pipeline triggers include cron, upstream jobs, etc.

pipeline {
  agent any
  triggers { cron('H */4 * * 1-5') }
  stages { /* ... */ }
}

5. Input

Interactive input can pause a pipeline and request user confirmation or parameters.

pipeline {
  agent any
  stages {
    stage('Example') {
      input {
        message "Continue?"
        ok "Proceed"
        submitter "alice,bob"
        parameters { string(name: 'PERSON', defaultValue: 'Mr Jenkins') }
      }
      steps { echo "Hello, ${PERSON}, nice to meet you." }
    }
  }
}

6. When

The when directive conditionally executes a stage based on branch, environment, expression, etc.

pipeline {
  agent any
  stages {
    stage('Example Deploy') {
      when { branch 'main' }
      steps { echo 'Deploying' }
    }
  }
}

2.3 Parallel

Parallel stages run concurrently.

pipeline {
  agent any
  stages {
    stage('Parallel Stage') {
      failFast true
      parallel {
        stage('Branch A') { steps { echo "On Branch A" } }
        stage('Branch B') { steps { echo "On Branch B" } }
        stage('Branch C') {
          stages {
            stage('Nested 1') { steps { echo "In Nested 1" } }
            stage('Nested 2') { steps { echo "In Nested 2" } }
          }
        }
      }
    }
  }
}

3. Using Jenkinsfile

Jenkinsfiles are usually stored in source control to enable versioning, review, and collaboration.

Facilitates code review of pipeline definitions.

Provides audit trails.

Allows multiple team members to edit the pipeline.

3.1 Environment Variables

1. Static Variables

Built‑in variables such as BUILD_ID, BUILD_NUMBER, BUILD_TAG, BUILD_URL, JOB_NAME, NODE_NAME, JENKINS_URL, and WORKSPACE are available via env.

2. Dynamic Variables

Variables can be set from command output using returnStdout or returnStatus.

// Jenkinsfile (Declarative Pipeline)
pipeline {
  agent any
  environment {
    CC = "${sh(returnStdout: true, script: 'echo -n "clang"')}"
    EXIT_STATUS = "${sh(returnStatus: true, script: 'exit 1')}"
  }
  stages {
    stage('Example') {
      environment { DEBUG_FLAGS = '-g' }
      steps { sh 'printenv' }
    }
  }
}

3.2 Credential Management

The credentials() function supports secret text, username/password, and secret files.

1. Secret Text

pipeline {
  agent any
  environment {
    AWS_ACCESS_KEY_ID = credentials('txt1')
    AWS_SECRET_ACCESS_KEY = credentials('txt2')
  }
  stages { /* ... */ }
}

2. Username/Password

pipeline {
  agent any
  environment { BITBUCKET_COMMON_CREDS = credentials('harbor-account') }
  stages { stage('printenv') { steps { sh "env" } } }
}

This creates three variables: BITBUCKET_COMMON_CREDS (username:password), BITBUCKET_COMMON_CREDS_USR, and BITBUCKET_COMMON_CREDS_PSW.

3. Secret File

// Jenkinsfile (Declarative Pipeline) using a Kubernetes pod
pipeline {
  agent {
    kubernetes {
      cloud 'kubernetes'
      slaveConnectTimeout 1200
      workspaceVolume emptyDirWorkspaceVolume()
      yaml '''
kind: Pod
metadata:
  name: jenkins-agent
spec:
  containers:
  - args: ['$(JENKINS_SECRET)', '$(JENKINS_NAME)']
    image: '192.168.10.15/kubernetes/jnlp:alpine'
    name: jnlp
    imagePullPolicy: IfNotPresent
  - command: ['cat']
    image: '192.168.10.15/kubernetes/kubectl:apline'
    name: kubectl
    tty: true
  restartPolicy: Never
''' 
    }
  }
  environment { MY_KUBECONFIG = credentials('kubernetes-cluster') }
  stages {
    stage('kubectl') {
      steps { container(name: 'kubectl') { sh "kubectl get pod -A --kubeconfig $MY_KUBECONFIG" } }
    }
  }
}

Original article link: https://zhangzhuo.ltd/articles/2022/06/04/1654333399919.html