Essential DevOps Tools for Operations: Packages, Monitoring, CI/CD and More

DevOps is a collection of processes, methods, and systems that promote communication, collaboration, and integration between development (application/software engineering), technical operations, and quality assurance (QA) teams. Using open‑source tools throughout the DevOps workflow can enhance project management and boost productivity.

The author, Richard Kraaijenhagen, founder of Owlin, full‑stack engineer and data scientist, gathered a wide range of tools that may be used in DevOps and categorized them by responsibility. Below is a selection of those tools.

Package & Product Management Tools

Chocolatey – an open‑source command‑line package manager for Windows, similar to apt‑get.

FPM – "Effing Package Management", a tool that simplifies building packages (deb, rpm, etc.) across multiple platforms.

Herd – a file distribution system based on Twitter Murder.

Vagrant Cachier – a Vagrant plugin for caching packages.

WiX Toolset – a powerful set of tools for creating Windows installers from XML source.

Boxstarter – automates software installation and creates repeatable, scripted Windows environments using Chocolatey.

Elita – an engine/framework for continuous deployment and API‑driven infrastructure using Git and Salt.

Fig – enables rapid, isolated development environments together with Docker.

Pulp – a platform for managing software repositories and related content.

Veewee – an open‑source tool for creating and configuring lightweight, reproducible virtual machine environments.

Logging & Monitoring

AmonOne – a modern self‑hosted server monitoring tool.

Anthracite – an application for event/log change management.

collectd3 – a visual collectd system performance statistics tool.

collectd – a daemon that gathers system performance data and stores it in various back‑ends.

Diamond – a Python‑based daemon that collects system metrics and publishes them to Graphite or other tools.

Errbit – an open‑source tool for collecting and managing application errors.

Sensu – an open‑source monitoring framework.

Logstash – a platform for transporting, processing, managing, and searching application logs and events.

log.io – a real‑time open‑source log monitoring tool.

FnordMetric – a Redis/Ruby‑based real‑time event tracking application for visualizing time‑series data.

Logster – reads log files and creates metrics usable by Graphite or Ganglia.

Kibana – a web interface for Logstash and Elasticsearch that enables efficient log search, visualization, and analysis.

Monit – a feature‑rich monitoring tool for processes, files, directories, and devices on Unix platforms, capable of automatic remediation.

Metrics – a lightweight detector written in Go (not a Java library).

Graphite – an open‑source project for collecting and visualizing real‑time website metrics.

Ganglia – a scalable distributed monitoring system for high‑performance computing clusters.

Server Density – a cross‑platform monitoring system.

Folsom – an Erlang‑based metrics system inspired by Coda Hale's metrics.

CMB (Cloud Message Bus) – a highly available, horizontally scalable queue and notification service compatible with AWS SQS and SNS.

Glances – an open‑source command‑line system monitoring tool for Linux/BSD written in Python.

Uptime – a remote monitoring system built with Node.js, MongoDB, and Twitter Bootstrap.

Icinga – an extension of Nagios.

Packetbeat – an open‑source application monitoring and packet tracing system.

Zipkin – an open‑source project from Twitter for collecting distributed tracing data.

Dead Man’s Snitch – a monitoring tool for Heroku Scheduler and cron jobs.

Statsd – a simple, lightweight Node.js daemon that sends metrics via UDP, often used with Graphite.

Riemann – a network monitoring system.

Puppet Dashboard – a web interface for Puppet providing node classification and reporting.

jmxtrans – a powerful tool for generating JSON‑based configurations and outputting them in various formats.

Scales – tracks server status and statistics, can send metrics to Graphite for visualization or write to files.

Zabbix – an enterprise‑grade, web‑based distributed monitoring solution.

Graylog 2 – stores system syslog data in MongoDB.

Process Management

Bouncy – can act as an HTTP routing host.

Supervisor – a client‑server system that allows users to monitor and control processes on Unix‑like operating systems.

God – a Ruby‑based process monitoring framework.

Service Discovery

Consul – simplifies service registration and discovery in distributed environments via HTTP or DNS interfaces, supporting external SaaS providers.

etcd – a highly available key/value store used for sharing configuration and service discovery.

Apache ZooKeeper – an Apache Hadoop sub‑project that solves common data management problems in distributed applications.

Weave – creates a virtual network that connects Docker containers deployed across multiple hosts.

Continuous Integration & Delivery

Buildbot – automates build and test cycles to verify code changes, reducing unnecessary failures.

Cabot – an open‑source self‑hosted monitoring tool.

Jenkins – a Java‑based continuous integration tool for automating repetitive tasks.

Hubot – a highly extensible scripting robot.

Hudson – an extensible CI engine for automated building and testing of software projects.

CruiseControl.rb – a CI server that keeps team members informed about project health and progress.

OpsBot – an open‑source, pluggable communication‑enhancing bot.

These tools aim to provide developers with practical assistance; the original source maintains an ever‑updating list of additional utilities. For further exploration, refer to the original article and the DevOps BookMarks collection.