Design and Implementation of Vivo Jenkins Scheduler for High Availability and Resource Management

The article discusses the limitations of traditional Jenkins deployments, which rely on a single master and suffer from resource imbalance, lack of automatic task rescheduling on slave failure, master overload, non‑persistent queues, and uncontrolled workspace growth.

It outlines several industry‑wide high‑availability approaches—Gearman + Jenkins, database‑backed file storage, and active‑passive standby modes—highlighting their advantages and drawbacks.

To overcome these shortcomings, Vivo designed a Jenkins Scheduler system with goals of improving build service reliability, reducing disaster recovery time, effectively distributing tasks, enabling rapid failover with administrator notifications, and providing visual analytics for build metrics.

The system adopts a fully master‑based Jenkins cluster and a custom scheduling platform. Key components include an API Gateway (handling authentication, routing, rate limiting, logging, and data transformation), an Event Center (using Spring’s event mechanism for Jenkins registration, failure detection, job redistribution, reception, and execution), a Scheduling Center (implementing Jenkins grouping by labels and a multi‑dimensional selection algorithm), flow‑control and queue management (using Redis and MySQL for request throttling and persistence), and a Callback Center (monitoring job states for notifications and build record storage).

Deployment in Vivo’s production environment has stabilized Jenkins operations, and future work aims to containerize and pool Jenkins instances to better align with Kubernetes‑based workflows.