Linux Performance Analysis Tools Overview

Motivated by a strong interest in Linux and low‑level system knowledge, the author aggregates a wide range of Linux performance analysis tools, drawing on Brendan Gregg’s performance tuning blog and other references to provide a practical, self‑contained guide.

Basic Monitoring Commands

vmstat – reports virtual memory, processes, CPU, and I/O statistics. Usage: vmstat interval times. The first line shows averages since boot; subsequent lines show current values at the specified interval.

iostat – reports CPU and device I/O statistics. Example output includes columns such as rrqm/s, wrqm/s, r/s, w/s, await, and %util. Common usage: iostat -x for extended device stats.

dstat – combines CPU, disk I/O, network, and paging information in a colorful, readable format. Example command: dstat -cdlmnpsy.

iotop – interactive top‑like display of per‑process disk I/O. Non‑interactive usage: iotop -bod interval.

pidstat – monitors resource usage of specific processes. Examples: pidstat -d interval for I/O, pidstat -u interval for CPU, pidstat -r interval for memory.

top / htop – real‑time process viewer. top shows load, process states, CPU, memory, and swap. htop adds color, mouse support, and easier scrolling; it starts faster and allows killing processes without typing PIDs.

mpstat – reports per‑CPU statistics from /proc/stat. Example: mpstat -P ALL interval times.

netstat – displays network connections and routing tables. Common flags: netstat -npl, netstat -rn, netstat -i.

ps – shows current process status. Typical usage: ps aux or ps -ef | grep. Examples for killing processes are also provided.

strace – traces system calls and signals of a program. Example: strace -e stat64 mysqld --print -defaults > /dev/null.

uptime – prints system uptime and load averages for the past 1, 5, and 15 minutes.

lsof – lists open files. Examples: lsof /boot, lsof -i :3306, lsof -u username, lsof -p 4838, lsof -i @192.168.34.128.

perf – kernel‑level performance analysis tool that samples program execution based on timer ticks, helping identify hot functions and cache‑miss ratios.

sar – System Activity Reporter, a comprehensive tool for reporting CPU, memory, I/O, and process activity. Usage: sar [options] [-A] [-o file] t [n] where t is the interval and n the count.

Advanced Performance Tools

perf_events – kernel‑maintained performance diagnostics, usable for both user‑space and kernel code.

eBPF tools – built with BCC for kernel tracing; maps are managed from user space via bpf system calls.

perf-tools – a lightweight collection based on perf_events and ftrace, supporting Linux 3.2+.

bcc (BPF Compiler Collection) – provides eBPF‑based tracing utilities; requires Linux 4.1+.

ktap – dynamic Linux kernel tracing script similar to DTrace/SystemTap.

Flame Graphs – visualizations generated from perf, systemtap, or ktap output; source at github.com/brendangregg/flamegraph.

Tool Classification

Basic tools include uptime, top / htop, mpstat, iostat, vmstat, free, ping, nicstat, and dstat. Advanced commands comprise sar, netstat, pidstat, strace, tcpdump, blktrace, iotop, slabtop, sysctl, and direct /proc inspection.

Summary

By matching each performance aspect with the appropriate Linux command, readers can systematically monitor and diagnose system behavior, from CPU and memory usage to I/O and network activity, and then apply advanced tracing and profiling tools for deeper analysis and optimization.