Master Linux System Performance: Top, perf, sar, vmstat, pidstat and More

01. top

top is a widely used tool to view system resource usage such as CPU and memory. It reads load averages from /proc/loadavg and displays the first three numbers as the 1‑, 5‑, and 15‑minute averages. The fourth field shows running processes over total processes, and the last field is the most recent PID. 9.53 9.12 8.37 3/889 28165 Use top -H -b -d 1 -n 200 > top.txt to capture thread details every second for 200 iterations.

1.1 /proc/loadavg

The three numbers represent average process counts over 1, 5, and 15 minutes, including running and ready processes.

1.2 top usage

Press H to show threads, 1 for per‑CPU usage, and use the -d option to set the update interval.

02. perf

The system‑level performance analysis tool perf focuses on CPU usage. Run sudo perf top -s comm to see the proportion of each process in total runtime. Record samples with sudo perf record and view them using sudo perf report -s comm.

03. sar and ksar

sar

(System Activity Report) monitors real‑time system activity and can generate historical reports. Install it with sudo apt install sysstat and enable it in /etc/default/sysstat. Use ksar to graph the data.

# Enable sar
enabled="true"
$ sudo gedit /etc/default/sysstat
# Configure cron for sar
$ sudo gedit /etc/cron.d/sysstat
# Restart sar service
$ sudo /etc/init.d/sysstat restart
# Save sar output
LC_ALL=C sar -A > sar.txt

Load the sar.txt file in ksar to generate graphs.

04. mpstat

mpstat

(Multiprocessor Statistics) shows average statistics since boot when run without arguments. Use mpstat -P ALL 10 20 to monitor all CPUs every 10 seconds for 20 iterations. $ mpstat -P ALL 10 20 The output columns include user, nice, system, iowait, irq, softirq, idle, guest, and gnice.

05. uptime

uptime

displays how long the system has been running and the load averages for the past 1, 5, and 15 minutes, reading data from /proc/uptime and /proc/loadavg.

11:15:41 up 82 days, 20:34, 8 users, load average: 0.28, 0.40, 0.43

06. vmstat

vmstat

primarily monitors memory usage but also provides CPU information. Run vmstat 5 5 to execute five samples at five‑second intervals.

procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
1  0 472576 228688 559092 1061756    0    0     9    39    1    0  8   4 87  0  0

The columns cover processes, memory, swap, I/O, system, and CPU statistics.

07. pidstat

pidstat

monitors resource usage per process. Use pidstat -p ALL to display all processes, -t for thread details, and options like -r for memory and -d for I/O.

# Show all process stats including idle
pidstat -p ALL
# Show thread stats
pidstat -p ALL -t
# Sample interval and count
pidstat 1 10

Sample output fields include minor/major page faults, VSZ, RSS, %MEM, and command name.

08. time

The time command measures CPU time of a program. Example:

real    0m0.263s
user    0m0.094s
sys     0m0.011s

Running the command again shows reduced I/O wait.

09. cpustat

Install with sudo apt install cpustat. Example output shows load average, CPU frequency, per‑process CPU usage, and distribution of CPU utilization.

Load Avg 0.66 0.54 0.49, Freq Avg. 1.46 GHz, 4 CPUs online
3791.1 Ctxt/s, 1709.9 IRQ/s, 1800.0 softIRQ/s
%CPU   %USR   %SYS   PID S  CPU   Time   Task
25.74 25.74  0.00 11435 R 3 2.29w /usr/bin/python3
...

10. htop

htop

offers a more readable interface than top. Press F5 to view threads in a tree structure.

11. atop

atop

monitors system resources and processes, sorting by CPU usage and displaying CPU, memory, disk, and network information.

12. glances

glances

(Python‑based) reports CPU, memory, network, disk, and process statistics, similar to nmon. Press h for help.

13. nmon

nmon

provides a single‑screen view of CPU, memory, network, disk, and process list, and can export data to spreadsheets.

13. pcp-gui

Performance Co‑Pilot (PCP) aggregates performance data from multiple hosts. Install with: $ sudo apt install pcp pcp-gui Open views such as CPU, Disk, Memory via the GUI menu.

14. collectl and colplot

14.1 collectl usage

collectl

gathers extensive performance data (CPU, disk, memory, network, sockets, TCP, etc.) and can replace tools like top, vmstat, and ps. Install with: sudo apt-get install collectl Basic usage displays CPU, disk, and network. Use options for detailed subsystems, e.g., -c for per‑CPU stats, -m for memory, -n for networks.

collectl --all   # all subsystems
collectl --top   # replace top
collectl --vmstat   # replace vmstat
collectl -c1 -sZ -i:1   # replace ps

14.2 colplot usage

colplot

visualizes collectl data in a browser. Download from the collectl‑utils project, install with sudo ./INSTALL, then reload Apache:

$ sudo systemctl reload apache2
$ sudo systemctl restart apache2

Access http://127.0.0.1/colplot/, select the directory containing collectl -P data, configure plot details, and generate the graph.