Master Kubernetes Jobs & CronJobs: A Practical Guide to Batch & Scheduled Tasks

Kubernetes Jobs Overview

Kubernetes Jobs are designed for short‑lived, batch workloads that run to completion, unlike Deployments, ReplicaSets, ReplicationControllers, or DaemonSets which run continuously.

A Job keeps running until the task defined in the pod finishes; if the pod exits with code 0 the Job succeeds. In normal Kubernetes behavior, a Deployment will replace failed pods, but a Job will simply retry according to its back‑off policy.

If the node hosting a Job pod fails, the pod is automatically rescheduled to another node.

Typical Use Cases

Batch processing tasks : run a job once daily or on a specific schedule, e.g., reading files from storage or a database and processing them.

Ops / ad‑hoc tasks : execute scripts such as database cleanup, cluster backup, or other maintenance activities.

Creating a Kubernetes Job

In this example we use an Ubuntu container that runs a shell script with a numeric argument controlling how many times the script echoes a message.

Steps:

Step 1: Create a job.yaml using a custom Docker image and pass the argument (e.g., 100) to the container’s entrypoint.

Step 2: Deploy the job with kubectl apply -f job.yaml.

Step 3: Check the job status with kubectl get jobs.

Step 4: List pods using kubectl get po.

Step 5: View the job pod’s logs with kubectl logs <pod-name> -f.

Running Jobs in Parallel

You can run a Job across multiple pods simultaneously by setting completions and parallelism in the manifest.

completions: 6
parallelism: 2

These fields tell Kubernetes to create six pods and run two of them at a time until all completions finish.

Generating Random Job Names

Creating multiple Jobs from the same manifest will cause a name conflict. To avoid this, add a generateName field in the metadata; Kubernetes will prepend the given prefix (e.g., kube-job-) and append a random suffix each time the manifest is applied.

Creating a Kubernetes CronJob

CronJobs let you schedule Jobs using standard cron syntax. For example, to run a job every 15 minutes you can use the schedule expression "0,15,30,45 * * * *".

Save the manifest as cron-job.yaml and deploy it with kubectl create -f cron-job.yaml. List CronJobs using kubectl get cronjobs and inspect logs by retrieving the associated pods.

Manually Triggering a CronJob

When you need to run a CronJob on demand, create a one‑off Job from the CronJob template:

kubectl create job --from=cronjob/kubernetes-cron-job manual-cron-job

This copies the CronJob’s spec and starts a Job named manual-cron-job.

Important Job and CronJob Parameters

failedJobHistoryLimit & successfulJobsHistoryLimit : limit how many failed or successful Job records are retained.

backoffLimit : maximum number of retries for a failed pod.

activeDeadlineSeconds : hard limit on how long a Job or CronJob may run (e.g., set to 60 for a one‑minute run).

By following this guide you can create, configure, and manage Kubernetes Jobs and CronJobs to automate batch processing and scheduled tasks efficiently.