Mastering Jenkins: Understanding Single‑Node vs Distributed Architectures

Jenkins Overview

Jenkins is an open‑source continuous integration (CI) tool with a user‑friendly interface, originating from Hudson. It automates building and testing of software projects and monitors external tasks.

Written in Java, Jenkins can run in servlet containers such as Tomcat or as a standalone service. It integrates with SCM tools, build tools (Maven, Ant, Gradle) and version control systems (SVN, Git), and can connect directly to GitHub.

Jenkins Deployment Architecture Overview

Jenkins can be deployed in two main modes:

Single‑Master installation: a sole Jenkins instance handles task management and build deployment, which may pose production security risks.

Master‑plus‑Agent mode: the Master schedules jobs and distributes them to Agents for execution, while monitoring Agent status.

Both Master and Agent can be installed on virtual machines or containers, and various combinations are possible.

Jenkins Deployment Architecture Details

3.1 Single‑Node Architecture (Single Master)

A single Master installation manages tasks and builds directly. For specific scenarios, a single node may be insufficient:

If all executors are busy, jobs must wait.

Increasing project count or load can exhaust resources.

Scaling a single node requires adding CPU or memory, which lacks elasticity and leads to downtime during upgrades. It also results in idle resources and security concerns because the Jenkins user has full access to all workspaces.

Therefore, Jenkins supports a distributed architecture where workload is delegated to multiple agents.

3.2 Distributed Architecture (Master‑Agent)

The Master (Controller) schedules build jobs and dispatches them to Agents, monitoring their status. Agents execute the assigned tasks and return results.

Both Master and Agents can run on VMs or containers, allowing flexible deployment choices.

Agent Types

Fixed Agent

A fixed Agent runs continuously; after a build it is not destroyed, thus continuously occupying cluster resources. Each Agent can host multiple executors, configured based on host resources.

Linux Jenkins (common)

Windows Jenkins

Container‑based agents

Note: Many build steps run shell commands, so the VM or container must have the required tools (e.g., Maven) installed.

Dynamic Agent

Dynamic agents are created as containers at build time and destroyed after the job finishes, offering high resource utilization but a more complex setup. Each dynamic agent can be treated as a dynamic executor.

Dependencies include cloud integration: the Jenkins Controller interacts with cloud providers via plugins to create and destroy resources (VMs or containers) for agents.

Docker Plugin – creates containers on a configured Docker host.

Kubernetes Plugin – creates Pods on a configured Kubernetes cluster.

Note: The Controller itself can be deployed inside or outside Kubernetes.

3.2.2 Characteristics of Jenkins Distributed Structure

Master (Controller) responsibilities:

Schedule build jobs.

Dispatch jobs to agents for execution.

Monitor agents and stop them when needed.

Agent (Slave) responsibilities:

Receive and execute tasks from the Master.

Optionally bind specific jobs to particular agents.

Handle actual build execution while the Master manages HTTP requests and environment setup.

This configuration enables horizontal scaling of Jenkins, allowing the system to grow by adding more agents while keeping the core installation on a single node.