Using Python Scripts for Operations Automation: Remote Execution, Log Parsing, Monitoring, Deployment, and Backup

Many operations engineers use Python scripts to automate routine tasks because Python offers a rich ecosystem of third‑party libraries and strong automation capabilities applicable to various domains.

Typical automation tasks include connecting to remote servers and executing commands, parsing log files, monitoring system status with alerts, batch deploying software or updates, and performing backup and recovery operations.

1. Connect to remote servers and execute commands

Python can use the paramiko library to establish SSH connections and run commands on remote hosts.

<code>import paramiko

# Create SSH client
ssh = paramiko.SSHClient()

# Auto‑accept unknown host keys
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())

# Connect to remote server
ssh.connect(hostname='remote.server.com', username='user', password='password')

# Execute command
stdin, stdout, stderr = ssh.exec_command('ls -l /tmp')
</code>

2. Parse log files and extract useful information

The regex library provides powerful regular‑expression tools for extracting data from logs.

<code>import regex

# Read log file
with open('log.txt', 'r') as f:
    log = f.read()

# Find error messages
errors = regex.findall(r'ERROR:\s+(.*)', log)

# Print matched errors
for error in errors:
    print(error)
</code>

More advanced parsing can use libraries such as loguru or python‑logstash .

3. Monitor system status and send alerts

The psutil library can retrieve CPU, memory, disk, and network metrics, while smtplib can send email alerts.

<code>import psutil
import smtplib

cpu_percent = psutil.cpu_percent()
if cpu_percent > 80:
    server = smtplib.SMTP('smtp.example.com')
    server.login('user', 'password')
    message = f'CPU usage exceeds 80%: {cpu_percent}%'
    subject = 'Alert: High CPU Usage'
    server.sendmail('[email protected]', '[email protected]', subject, message)
    server.quit()
</code>

Thresholds and monitored resources can be customized, and other tools like nagios‑api or sensu‑client may be used.

4. Batch deploy software or update systems

The fabric library enables remote command execution across multiple servers.

<code>from fabric import task

@task
def update_system(c):
    c.run('apt-get update')
</code>

Alternatives such as Ansible or Puppet provide more extensive deployment capabilities.

5. Perform backup and recovery tasks

The standard shutil module can copy individual files or entire directories.

<code>import shutil

# Backup a single file
shutil.copy('/path/to/file', '/path/to/backup/file')
</code>

<code>import shutil

# Backup an entire directory
shutil.copytree('/path/to/dir', '/path/to/backup/dir')
</code>

Beyond these examples, Python can also be used for automated testing (e.g., pytest , selenium ), data analysis and visualization ( numpy , pandas , matplotlib ), and machine‑learning tasks ( scikit‑learn , tensorflow , nltk ), further extending its value for operations engineers.