Jenkins Distributed Builds: Adding Agent Nodes with Docker Compose

Theory: Why Distributed Builds and Core Concepts

Distributed builds offload execution from the Jenkins Controller to Agent nodes, providing resource isolation, environment specialization (e.g., specific JDK, Maven, Node.js, Docker), and improved stability through clear separation of scheduling and execution duties.

Key Agent Concepts

Node : A machine (physical, VM, or container) that Jenkins can schedule jobs on.

Label : Abstract capability tags like maven, docker, frontend.

Executor : Number of concurrent tasks a node can run.

Remote root directory : Agent workspace path, e.g., /home/jenkins/agent.

Common Connection Methods

Inbound Agent (recommended) : Agent initiates connection to Controller; suits containerized and restricted networks.

SSH Agent : Controller SSHes into target machine to start Agent; fits existing Linux hosts.

Jenkins Default UI Connection

The node detail page shows three command-line variants (Unix, Windows, secret-file) that all boil down to: download agent.jar, run java -jar agent.jar with parameters -secret, -name, -webSocket, -workDir. This native method works on any host but requires manual tool management.

Docker Compose as Recommended Practice

Because the Jenkins Controller already runs via docker-compose, managing Agents the same way offers unified configuration, easy rebuild/migration/scaling, and simple volume mounts for JDK, Maven, Node.js, Docker CLI, and custom scripts. The article distinguishes:

Jenkins default : agent.jar + java -jar native command line.

Article recommendation : docker-compose launching an Inbound Agent container with parameters from the Jenkins UI.

Practice: Adding an Agent for the k8s-dashboard Maven Project

0) Preparation: Create and Connect Agent

1. Create Node in Jenkins

Navigate: Manage Jenkins → Manage Nodes and Clouds → New Node. Configure:

Name: agent-1 Type: Permanent Agent Executors: 1 Remote root directory: /home/jenkins/agent Labels: maven Usage: Use this node as much as possible Launch method: Launch agent by connecting it to the controller (Inbound)

Save, then note the Agent name ( agent-1) and auto-generated Secret .

2. Jenkins Default Command Line (for quick verification)

curl -sO http://192.168.145.129:8080/jenkins/jnlpJars/agent.jar
java -jar agent.jar \
  -url http://192.168.145.129:8080/jenkins/ \
  -secret <Secret> \
  -name "agent-1" \
  -webSocket \
  -workDir "/home/jenkins/agent"

Suitable for existing Linux/Windows hosts without Docker.

3. Docker Compose Management (Recommended)

Step 1: Prepare directories

mkdir -p ~/jenkins-agent-demo
cd ~/jenkins-agent-demo

Step 2: Create workspace and tool directories

mkdir -p ./agent-data/agent-1
mkdir -p ./tools/jdk
mkdir -p ./tools/maven
mkdir -p ./tools/node
mkdir -p ./tools/git

Step 2.1: Fix permissions (critical because jenkins/inbound-agent runs as non-root UID 1000)

chmod -R 777 ./agent-data/agent-1
# Or more properly:
chown -R 1000:1000 ./agent-data/agent-1
chmod -R 755 ./agent-data/agent-1

Step 3: Confirm Jenkins UI values — Agent Name, Secret, Jenkins URL ( http://192.168.145.129:8080/jenkins/), WorkDir ( /home/jenkins/agent).

Step 4: Create docker-compose-agent.yml

version: '3.8'

services:
  agent-1:
    image: jenkins/inbound-agent:jdk17
    container_name: agent-1
    restart: unless-stopped
    environment:
      JENKINS_URL: http://192.168.145.129:8080/jenkins/
      JENKINS_AGENT_NAME: agent-1
      JENKINS_SECRET: <Secret>
      JENKINS_AGENT_WORKDIR: /home/jenkins/agent
    volumes:
      - ./agent-data/agent-1:/home/jenkins/agent
      - ./tools/jdk:/opt/tools/jdk:ro
      - ./tools/maven:/opt/tools/maven:ro
      - ./tools/node:/opt/tools/node:ro
      - ./tools/git:/opt/tools/git:ro
      - /var/run/docker.sock:/var/run/docker.sock
      - /usr/bin/docker:/usr/bin/docker
    networks:
      - ci-network

networks:
  ci-network:
    driver: bridge

Verify JENKINS_SECRET, JENKINS_URL includes /jenkins/, and JENKINS_AGENT_NAME matches.

Step 5: Start Agent

docker compose -f docker-compose-agent.yml up -d
# or legacy
docker-compose -f docker-compose-agent.yml up -d

Step 6: Check container docker ps --filter "name=agent-1" Step 7: View logs docker logs -f agent-1 — look for Connected and Jenkins agent is running. Jenkins UI should show agent-1 as Online.

Troubleshooting permission error : If logs show

The specified working directory should be fully accessible to the remoting executable (RWX): /home/jenkins/agent

, re-run permission fix and restart.

Step 8: Maintenance commands

Stop: docker compose -f docker-compose-agent.yml down Restart: docker compose -f docker-compose-agent.yml up -d Rebuild: down then up

Logs: docker logs --tail 100 agent-1 Step 9: Adjust tool mount paths — left side is host path, right side is container path (read-only :ro).

Step 10: Minimal verification — temporarily remove tool mounts to first verify connectivity, then add them back.

4. Recommended Tool Mounts

./agent-data/agent-1:/home/jenkins/agent

— persistent workspace/cache ./tools/jdk:/opt/tools/jdk:ro — unified JDK ./tools/maven:/opt/tools/maven:ro — unified Maven ./tools/node:/opt/tools/node:ro — for frontend builds ./tools/git:/opt/tools/git:ro — optional unified Git /var/run/docker.sock:/var/run/docker.sock — Docker-in-Docker /usr/bin/docker:/usr/bin/docker — host Docker CLI

Principles: tools read-only, workspace persistent, Docker capability only if pipelines build images.

5. Single docker run for Quick Test

docker run -d --name agent-1 --restart=always \
  -e JENKINS_URL=http://192.168.145.129:8080/jenkins/ \
  -e JENKINS_AGENT_NAME=agent-1 \
  -e JENKINS_SECRET=<Secret> \
  -e JENKINS_AGENT_WORKDIR=/home/jenkins/agent \
  -v jenkins_agent1:/home/jenkins/agent \
  jenkins/inbound-agent:jdk17

Good for one-off verification; switch to docker-compose for production.

6. Verify Connection

docker logs agent-1 | grep -E "Connected|Jenkins agent is running" || true

Jenkins node page must show Online.

7. Confirm Global Tools & Plugins

Manage Jenkins → Global Tool Configuration

: JDK JDK11, Maven Maven3.9.10.

Required plugins: Pipeline Maven Integration (for withMaven in shared library), SonarQube Scanner, Email Extension.

Configure Git Host Key Verification → Manually provided keys with ssh-keyscan -t ed25519 -p 2222 192.168.145.129 output to avoid Host key verification failed on Agent.

8. Method Selection Guide

Daily experiments & long-term : docker-compose Temporary connectivity test : docker run Traditional hosts, no containers : Jenkins default command line

Three Ways to Run Builds on the Agent

Method 1: Freestyle Job (verify scheduling)

New Freestyle job k8s-dashboard-freestyle-agent.

Restrict to label maven.

Git: ssh://[email protected]:2222/root/k8s-dashboard.git, credentials gitlab-ssh-cred, branch */main.

Build step Execute shell:

echo "NODE_NAME=$NODE_NAME"; echo "WORKSPACE=$WORKSPACE"; java -version

.

Build step Invoke top-level Maven targets: Maven Maven3.9.10, goals clean package -DskipTests.

Success indicator in console:

Building remotely on agent-1 (maven) in workspace /home/jenkins/agent/workspace/k8s-dashboard-freestyle-agent

.

Method 2: Pipeline Job (recommended)

New Pipeline job k8s-dashboard-pipeline-agent, Pipeline script from SCM, same Git config, script path Jenkinsfile.

Modify existing Jenkinsfile: change agent any to agent { label 'maven' }.

Example pipeline includes stages: checkout, build ( mvn -B -U clean package -DskipTests), tool verification, archive artifacts, post notifications.

Success log starts with Running on agent-1.

Method 3: Shared Library my-shared-lib@main (enterprise pattern)

Configure Global Pipeline Library: name my-shared-lib, default version main, Modern SCM, Git ssh://[email protected]:2222/root/shared-library.git, credentials gitlab-ssh-cred.

Keep shared library logic unchanged; only change Jenkinsfile agent directive to agent { label 'maven' }.

Sample Jenkinsfile uses @Library('my-shared-lib@main') _, calls mavenBuild() and sonarScan() from library, includes branch strategy and post actions.

All identifiers (library name, credentials, tool names, repo URLs) stay consistent with previous chapters.

Verification & Troubleshooting

1. Node Online Status

Check Manage Nodes — agent-1 must be Online.

2. Agent Connection Failure

Check docker logs agent-1, verify JENKINS_URL and Secret. Permission error RWX: /home/jenkins/agent → fix mount permissions as in Step 2.1.

3. Git Host Key Verification Failed

Error: No ED25519 host key is known for [192.168.145.129]:2222. Fix: run ssh-keyscan -t ed25519 -p 2222 192.168.145.129, paste output into

Manage Jenkins → Security → Git Host Key Verification Configuration → Manually provided keys

.

4. Pipeline Still Runs on Controller

Ensure Jenkinsfile has agent { label 'maven' } and node label is exactly maven.

5. withMaven Errors

Verify Pipeline Maven Integration plugin installed, and global tools JDK11 & Maven3.9.10 exist.

6. Shared Library Load Failure

Check library name my-shared-lib, default version main, repo URL, credentials.

7. Standardize gitCheckout Parameter

Use credId: 'gitlab-ssh-cred' consistently.

8. Keep Documentation & Library in Sync

Function names and parameters in docs must match actual shared library implementation.

Summary & Recommendations

Agent integration is not a new workflow but an execution-location shift: keep existing pipelines, shared library, repos, credentials, tool names; only move execution from Controller to Agent.

Progression by difficulty:

Freestyle + Agent — verify connectivity (first step).

Pipeline + Agent — continue with existing Jenkinsfile (recommended).

Pipeline + Shared Library + Agent — best alignment with future chapters (most recommended).

Adopt the third pattern: Jenkinsfile as project entry, Shared Library for common logic, Agent as execution node. This structure remains stable for upcoming SonarQube, Docker builds, image push, multi-branch strategies, etc. Agent value lies in making existing pipelines more stable and scalable, not in complicating them.