How to Build Low‑Cost Automated Ops with Prometheus, Ansible & Jenkins

Background

A three‑and‑a‑half‑person development team maintains dozens of cloud machines and dozens of legacy applications. With limited resources they need a low‑cost way to automate operations.

Step 1: Monitoring and Alerting

After evaluating Zabbix, Open‑Falcon and Prometheus, the team chose Prometheus because it uses a pull model, stores configuration as plain text (easy to version), and offers many ready‑made exporters.

Deployment is automated with Ansible using the existing prometheus‑ansible role.

Prometheus Server – collects and stores metrics.

Alertmanager – evaluates alert rules and forwards alerts.

node‑exporter – exposes host metrics via HTTP.

Metrics are visualized in Grafana, which integrates seamlessly with Prometheus.

For alert notifications the team added a DingTalk webhook via the open‑source prometheus-webhook-dingtalk component.

Step 2: Configuration Versioning

All monitoring configuration is kept in a separate Git repository. Future plans include migrating to Consul as a configuration centre, which is already supported by Ansible 2.0+.

├── environments/
│   ├── dev/
│   │   ├── group_vars/
│   │   │   ├── all
│   │   │   ├── db
│   │   │   └── web
│   │   └── hosts
│   ├── prod/
│   │   ├── group_vars/
│   │   │   ├── all
│   │   │   ├── db
│   │   │   └── web
│   │   └── hosts
│   └── stage/
│       ├── group_vars/
│       │   ├── all
│       │   ├── db
│       │   └── web
│       └── hosts

Step 3: CI/CD with Jenkins

Jenkins is installed via an Ansible role ( ansible‑role‑jenkins) that also auto‑installs required plugins using the jenkins_plugins variable.

---
- hosts: all
  vars:
    jenkins_plugins:
      - blueocean
      - ghprb
      - greenballs
      - workflow-aggregator
    jenkins_plugin_timeout: 120
  pre_tasks:
    - include_tasks: java-8.yml
  roles:
    - geerlingguy.java
    - ansible-role-jenkins

Jenkins integrates with the existing GitLab instance and builds projects using a Jenkinsfile (pipeline syntax) stored alongside the source code.

pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                sh './gradlew clean build'
                archiveArtifacts artifacts: '**/target/*.jar', fingerprint: true
            }
        }
    }
}

The Jenkinsfile is placed in each project repository, allowing a separate pipeline job per project.

Step 4: Running Ansible from Jenkins

The Jenkins Ansible plugin executes playbooks. Credentials and vault passwords are supplied via the withCredentials block.

withCredentials([sshUserPrivateKey(keyFileVariable:"deploy_private",credentialsId:"deploy"),
                file(credentialsId: 'vault_password', variable: 'vault_password')]) {
    ansiblePlaybook vaultCredentialsId: 'vault_password',
                    inventory: "environments/prod",
                    playbook: "playbook.yaml",
                    extraVars:[
                        ansible_ssh_private_key_file: [value: "${deploy_private}", hidden: true],
                        build_number: [value: "${params.build_number}", hidden: false]
                    ]
}

Each project contains an ansible directory with its own playbooks; during deployment the directory is zipped, transferred, and unpacked on the target machines.

Step 5: Scaling the Process

To avoid repetitive manual work, the team uses cookiecutter to generate Jenkinsfiles and Ansible scaffolding for new projects automatically.

Summary Checklist

Deploy basic monitoring (Prometheus + Grafana).

Set up GitLab.

Install Jenkins and integrate with GitLab.

Implement automated build & packaging via Jenkins pipelines.

Execute Ansible playbooks from Jenkins for deployment.

From this foundation the team can evolve toward more advanced architectures such as CMDB generation with ansible‑cmdb, sophisticated release strategies, auto‑scaling via Prometheus alerts, and ChatOps integrations.