Python Tool for Automating Multi-Server Commands and rsync-Style File Sync

Python, invented by Guido van Rossum, is an object‑oriented scripting language that can serve as a full‑stack development tool, enabling work on front‑end, back‑end, testing, big data analysis, and web crawling.

Motivated by the need to standardize a fleet of servers, the author created a Python 3.5 utility that automates system‑level and application‑level tasks across more than 30 machines.

Batch execution of remote commands and file upload/download. Multithreaded concurrent execution to reduce waiting time for long‑running commands or transfers. Strict mode (stop on first error) and non‑strict mode. rsync‑like synchronization that transfers only files with differing modification time or size. Rich command‑line prompts. Cross‑platform support for both Linux and Windows.

The tool relies on the external packages docopt for the command‑line interface and paramiko for SSH and SFTP operations. Server information is read from a file where each line follows the format “hostname‑IP:port”, with support for comment lines prefixed by #.

Paramiko’s SFTP client is extended to preserve file timestamps, transfer entire directories, and implement the rsync‑style check of mtime and size. Path handling automatically determines source and destination formats, so trailing slashes are not required when transferring directories.

An optional --skip-err flag allows the script to continue on servers that raise errors, such as missing commands during a bulk ls operation. Comprehensive error handling provides clear messages for almost every possible failure.

Key helper utilities include print_color() for colored terminal output on Linux and the OutputText class, which simplifies printing results from multithreaded tasks.

Below are screenshots of the help information, batch command execution, file upload, and file download interfaces.

The author notes that while similar tasks can be performed with shell scripts, SSH, or rsync, using Python provides a cross‑platform solution with built‑in multithreading, making it especially useful when coordinating between Windows clients and Linux servers.

Developing the script also deepened the author's understanding of SSH login processes, threading models, and the synergy between operations and development.