How to Build Custom Jenkins Agent AMIs with Packer for Scalable CI/CD

Overview

Jenkins can launch build agents on Amazon EC2 through the AWS EC2 plugin. To ensure every agent starts with the same toolchain—Docker and JDK 17 in this example—a custom Ubuntu AMI is built with HashiCorp Packer and referenced in the plugin configuration.

What is Packer?

Packer is an open‑source image‑building tool that creates immutable machine images for many cloud providers from a single HCL template. The template describes the base operating system, provisioners that install software, and the resulting artifact (e.g., an AWS AMI).

Why use Packer for Jenkins agents?

Automation : The entire image build is scripted; no manual instance setup is required.

Consistency : All agents are launched from the same AMI, eliminating configuration drift.

Customizability : Tools such as Docker and JDK can be pre‑installed.

Direct Jenkins integration : The AWS EC2 plugin can launch agents from the Packer‑generated AMI.

Packer template (HCL)

packer {
  required_plugins {
    amazon = {
      version = ">= 1.2.8"
      source  = "github.com/hashicorp/amazon"
    }
  }
}

source "amazon-ebs" "ubuntu" {
  ami_name      = "ubuntu-24.04-docker-jdk17-${formatdate(\"YYYYMMDD\", timestamp())}"
  instance_type = "t3.small"
  region        = "ap-southeast-1"
  source_ami_filter {
    filters = {
      name                = "ubuntu/images/*ubuntu-noble-24.04-amd64-server-*"
      root-device-type    = "ebs"
      virtualization-type = "hvm"
    }
    most_recent = true
    owners      = ["099720109477"]
  }
  ssh_username = "ubuntu"
}

build {
  name    = "build-docker-ami"
  sources = ["source.amazon-ebs.ubuntu"]
  provisioner "shell" {
    inline = [
      "sudo apt-get update",
      "sudo apt-get install -y ca-certificates curl gnupg",
      "sudo install -m 0755 -d /etc/apt/keyrings",
      "sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg",
      "sudo chmod a+r /etc/apt/keyrings/docker.gpg",
      "echo \"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable\" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null",
      "sudo apt-get update",
      "sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin",
      "sudo apt-get install -y openjdk-17-jdk",
      "docker --version",
      "java -version"
    ]
  }
}

Template explanation

Source configuration : Uses the latest Ubuntu 24.04 AMI in the ap-southeast-1 region with instance type t3.small.

Provisioning steps : A shell provisioner updates the package index, adds Docker’s apt repository, installs Docker Engine and JDK 17, then prints version information to verify the installation.

AMI naming : Includes a date stamp so each build produces a unique image name.

Building the AMI

Execute the following command in the directory containing the template file (e.g., my-agent.pkr.hcl): packer build my-agent.pkr.hcl When the build finishes, the new AMI appears in the AWS console. Record the AMI ID (for example ami-01e29b32884522433) for later use in Jenkins.

Integrating the custom AMI with Jenkins AWS EC2 plugin

1. Install the plugin

From Jenkins’ Plugin Manager, install AWS EC2 Plugin .

Navigate to Manage Jenkins → Configure System and add a new Amazon EC2 cloud.

Provide AWS credentials (Access Key and Secret Key) and select the same region used in the Packer template (e.g., ap-southeast-1).

2. Configure the custom AMI

Enter the AMI ID produced by Packer in the AMI ID field.

Set the instance type ( t3.small), SSH username ( ubuntu), security groups, and any required subnet.

Assign a label (e.g., docker-jdk17) that pipelines will reference.

3. Define the agent template

Select Unix as the AMI type.

Set Remote FS root to a directory such as /home/ubuntu.

Configure an idle termination timeout (e.g., 30 minutes) so unused agents are automatically terminated.

4. Save and test

After saving, Jenkins can launch an EC2 instance from the custom AMI on demand. Verify the launch by checking the EC2 console or by running a simple job.

Using the custom agent in a Jenkins pipeline

pipeline {
    agent { label 'docker-jdk17' }
    stages {
        stage('Build') {
            steps {
                sh 'java -version'
                sh 'docker --version'
                echo 'Running on a custom AMI with Docker and JDK 17!'
            }
        }
    }
}

The pipeline requests an agent with the docker-jdk17 label; Jenkins starts an EC2 instance from the Packer‑built AMI, runs the steps, and then terminates the instance automatically.

Advantages of this approach

Dynamic scaling : Agents are created and destroyed on demand, matching workload.

Environment consistency : All agents share the same pre‑installed Docker and JDK 17, eliminating “works on my machine” issues.

Cost optimization : Pay‑as‑you‑go usage; idle instances are terminated after the configured timeout.

Maintainability : The Packer template is version‑controlled; updating the toolchain only requires rebuilding the AMI and updating the AMI ID in Jenkins.

Conclusion

By building a custom Ubuntu AMI with Docker and JDK 17 using HashiCorp Packer and wiring that AMI into Jenkins’ AWS EC2 plugin, teams obtain a reproducible, cost‑effective CI/CD environment where every build agent starts with the exact required software stack.