Essential Linux Performance Monitoring Tools and How to Use Them

Performance Analysis Tools Overview

The article compiles a wide range of Linux observability and benchmarking tools, providing brief usage instructions and illustrating their output with screenshots. These utilities help users monitor memory, CPU, I/O, network, and process behavior, enabling detailed performance diagnosis.

1. vmstat – Virtual Memory Statistics

vmstat reports virtual memory, processes, CPU, and I/O statistics. Usage: vmstat interval times (e.g., vmstat 5 10). The first line shows averages since boot; subsequent lines show real‑time data at the specified interval. Key columns include procs (r/b), memory (swapd, free, buffers, cache), swap , io , and cpu (us, sy, id, wa, st).

2. iostat – CPU and Device I/O Statistics

iostat displays CPU usage and per‑device I/O metrics. Use iostat -x interval times for extended device statistics. Important fields: rrqm/s, wrqm/s, r/s, w/s, rsec/s, wsec/s, avgrq‑sz, avgqu‑sz, await, svctm, %util.

3. dstat – Versatile System Monitoring

dstat combines CPU, disk I/O, network, and paging statistics in a colorful, readable format. Run dstat -cdlmnpsy to display a comprehensive view.

4. iotop – Real‑Time Disk I/O per Process

iotop works like top but focuses on I/O. Example: iotop -bod interval shows I/O load per process.

5. pidstat – Per‑Process Resource Usage

pidstat monitors CPU, memory, I/O, and thread activity for specific processes. Examples: pidstat -d interval – I/O statistics pidstat -u interval – CPU usage pidstat -r interval – Memory usage

6. top / htop – Interactive Process Viewers

top provides a summary of load, processes, CPU, memory, and swap. htop adds color, mouse support, and horizontal/vertical scrolling, making it easier to navigate long command lines and kill processes without typing PIDs.

7. mpstat – Multiprocessor Statistics

mpstat reports per‑CPU statistics from /proc/stat. Example: mpstat -P ALL interval times shows each CPU’s activity.

8. netstat – Network Connections and Routing

netstat displays IP, TCP, UDP, and ICMP statistics, as well as routing tables. Common usage: netstat -npl (list listening ports) and netstat -rn (print routing table).

9. ps – Process Status

ps offers extensive options; a typical snapshot is ps aux. To kill a process: ps aux | grep mysqld | awk '{print $2}' | xargs kill -9. To kill zombie processes: ps -eal | awk '{if ($2 == "Z") print $4}' | xargs kill -9.

10. strace – System Call Tracing

strace traces system calls and signals. Example to see which config file mysqld loads: strace -e stat64 mysqld --print --defaults > /dev/null.

11. uptime – System Uptime and Load Averages

uptime prints how long the system has run and the 1‑, 5‑, and 15‑minute load averages.

12. lsof – List Open Files

lsof enumerates open files and the processes using them. Useful commands: lsof /boot – list files under /boot lsof -i :3306 – find process using port 3306 lsof -u username – files opened by a user lsof -p 4838 – files opened by PID 4838

13. perf – Kernel Performance Analyzer

perf, bundled with the Linux kernel, samples program execution to identify hot functions and cache misses. It works by triggering sampling at timer ticks and recording the execution context, allowing developers to pinpoint performance bottlenecks.

Additional Observability and Benchmarking Tools

Beyond the basic utilities, the article mentions advanced tools such as perf_events , eBPF tools , perf‑tools , bcc , ktap , and Flame Graphs for deeper kernel tracing and visualization.

Basic Toolset

uptime, top/htop, mpstat, iostat, vmstat, free, ping, nicstat, dstat.

Advanced Commands

sar, netstat, pidstat, strace, tcpdump, blktrace, iotop, slabtop, sysctl, /proc.