Why Did My Jenkins Pipeline Suddenly Slow Down? A Kubernetes Storage Mystery Solved

Recently I encountered a puzzling issue: Jenkins pipeline stages that normally switch instantly now take about a minute each, even for simple echo commands.

I checked Jenkins logs, configuration, and Kubernetes parameters, even added resources, but nothing helped. After hours of searching and consulting colleagues, I remembered a previous Kubernetes cluster that was set up with an Ansible script.

The Ansible‑deployed cluster performed excellently, while my manually built cluster was slow. The key differences were:

One cluster was provisioned via an online Ansible script; the other was built entirely by hand.

Both use Ceph for storage, but the Ansible setup uses RBD, whereas the manual setup uses CephFS (the crucial point).

Architectural differences in the cluster design.

I suspected the CephFS issue because it requires Linux kernel 4.10+, while my servers run kernel 3.10.

To test, I switched to NFS on the manual cluster. The pipeline speed returned to normal, confirming the storage layer was the bottleneck.

Finally, I reverted to using Ceph RBD on the manual cluster, and performance improved dramatically.

In summary, the slowdown was not caused by the way the cluster was built or by the file system itself, but by an incorrect Ceph configuration. Using the proper Ceph backend (RBD) or switching to NFS resolves the issue.