How Monitoring and Template Deployment Supercharge Automated Operations

Background

With the rapid development of the information age, IT operations have become a crucial part of IT services. As cloud computing and big data mature, automated production operations have risen to prominence. Traditional manual methods for large clusters—such as daily backups, status monitoring, and alerts—are inefficient, creating an urgent need for automation.

Traditional operation drawbacks include:

Human‑initiated events lead to passive, low‑efficiency operations.

High system heterogeneity lacks efficient processes.

The explosion of cloud and big‑data workloads makes existing tools insufficient.

Automation should follow four principles: systematic management , process workflow , professional personnel , and task automation .

Monitoring as the Core Concept of Automated Operations

Low efficiency stems from slow response times; operators stare at alert pages and wait for failures before notifying the right people. Therefore, server status monitoring must drive automated operations. The platform uses

ElkStack

,

Zabbix

, and

Zabbix‑Agent

to collect server health data and generate time‑series charts for analysis.

Effective alert strategies ensure that professionals handle professional issues. The system supports email, WeChat, and SMS alerts, notifying the appropriate personnel based on fault type and severity, with SLA‑based escalation. Future work includes extending WeChat integration with templated handling mechanisms.

For example, when a server’s disk usage reaches 90%, an alert is sent via WeChat; the on‑call engineer selects a cleanup template (e.g., data repair or log purge) and executes the task automatically.

Template‑Based Deployment as an Essential Automation Tool

For many engineers, the most time‑consuming part of operations is environment deployment, often consuming over 80% of effort due to OS version differences, manual initialization, and inconsistent software packages.

Using a template engine (e.g., Cobbler) standardizes OS installation, configuration, and package versions, ensuring identical base environments across the cluster and reducing deployment errors.

Beyond OS templating, combining

Ansible

Playbooks scripts common operational tools, enabling parallel configuration management and further reducing human error.

The platform’s two core components are the alarm scheduling engine (

messageserver

) and the event scheduling engine (

jobserver

). The alarm engine analyzes alerts, generating charts by event, time, machine, and category. The event engine automatically processes alerts to achieve true automation.

Automation Philosophy

Automation is not only about new tools but also about mindset and process improvement—a continuous improvement cycle.

First, operations must be standardized and workflow‑driven. Second, operational tools should be containerized or virtualized to minimize deployment time. Third, daily maintenance should be scripted using configuration‑management tools to reduce manual intervention. Finally, true automation integrates these elements into an intelligent, self‑maintaining system.

In practice, the most time‑consuming tasks involve workflow optimization, permission control, and log auditing. Adding jump‑host functionality further streamlines the platform.

Deploying a full environment now takes about twenty minutes; provisioning 100 servers simultaneously improves efficiency by 35×. Routine maintenance and application deployment rarely require direct node access—most actions are performed through the unified platform.

Ultimately, automated operations represent the evolution from Ops to DevOps, moving toward a fully intelligent operations era.

GOPS 2016 Global Operations Conference – Shenzhen