Master Linux System Monitoring: htop, top, load average, CPU, memory, and network metrics

CPU basics and /proc information

Linux treats everything as a file; you can view CPU details by reading /proc/cpuinfo. Common queries include the number of CPUs, model name, and clock speed. The command cat /proc/cpuinfo displays this information, while cat /proc/stat shows overall CPU statistics.

Load average

Use uptime or w to print the system load averages for the past 1, 5, and 15 minutes. The sar -q command provides dynamic load data. Load average represents the average number of processes in runnable or uninterruptible states; a value higher than the number of CPU cores (e.g., >4 on a 4‑core system) indicates overload.

Dynamic load monitoring

$ sar -q 1 100

The output lists run queue size, process list size, and load averages at 1‑second intervals, ending with an overall average.

CPU utilization

Both htop and top display CPU usage. The idle time can be derived from the top summary line. CPU utilization is calculated as 1 - idle_time / total_time. Example output:

%Cpu(s): 7.4 us, 2.3 sy, 0.0 ni, 90.1 id, 0.0 wa, 0.0 hi, 0.2 si, 0.0 st

user: time spent in user mode (excluding nice)

system: kernel mode time

nice: low‑priority user time

idle (id)

iowait (wa)

irq (hi)

softirq (si)

steal (st)

System calls with strace

Inspect system calls of a process using strace -p <pid>. The -c flag aggregates call counts and CPU time.

# View system calls of PID 7477
strace -p 7477
# Show calls made by a command
strace cat index.js
# Summarize call statistics
strace -p 7477 -c

Memory inspection

Use free -h for overall memory stats. For per‑process memory, pidstat -r or htop are useful.

total        used        free      shared  buff/cache   available
Mem:           3.7G        682M        398M        2.1M        2.6G        2.7G
Swap:            0B          0B          0B

To view a specific process’s RSS, run ps -O rss <pid>:

PID   RSS S TTY     TIME COMMAND
2579 19876 S pts/10 00:00:03 node index.js

Process handling

Common tasks include locating a process by name ( pgrep -a node), by arguments ( pgrep -af ts-node), checking status ( cat /proc/122/status), and printing the parent tree ( pstree 122 -sap).

Process states

D – uninterruptible sleep (usually I/O)

R – running or runnable

S – interruptible sleep

T – stopped by job control

t – stopped by debugger

W – paging (obsolete)

X – dead (should not appear)

Z – zombie (terminated but not reaped)

Use top or htop to view state counts, e.g., total tasks, running, sleeping, stopped, zombie.

PID mapping between containers and host

To find the host PID for a container process, read /proc/<pid>/sched inside the container, which contains the global PID. Then on the host, run ps -fp <global_pid> to view details.

# Inside container (PID 122)
cat /proc/122/sched
# Host side
ps -fp 7477
UID   PID  PPID C STIME TTY          TIME CMD
root 7477 7161  0 Jul10 ?        00:00:38 node index.js

To locate the container owning a host PID, use Docker inspection:

# List containers with their host PIDs and IDs
docker ps -q | xargs docker inspect --format '{{.State.Pid}}, {{.ID}}' | grep 22932
# Or examine cgroup files
cat /etc/22932/cgroup

Swap and inode information

Check swap statistics with vmstat -s. List inode numbers using ls -lahi.

Network throughput and socket status

Bandwidth is the link’s maximum rate; throughput is the actual data transferred per second (e.g., KB/s, MB/s). Use ifconfig eth0 for interface details, sar -n DEV 1 for per‑second stats, or netstat -tanp / ss -s for socket summaries. Non‑zero Recv‑Q or Send‑Q indicates packet backlog.

# Example netstat output
Proto Recv-Q Send-Q Local Address          Foreign Address        State       PID/Program name
tcp        0      0 127.0.0.11:35283       0.0.0.0:*              LISTEN      -
tcp        0      0 192.168.112.2:37344    172.18.0.1:6379       ESTABLISHED 78/node
tcp        0      0 :::80                 :::*                  LISTEN      78/node

Database connection counts

PostgreSQL:

show max_connections;
select count(*) from pg_stat_activity;

MySQL:

show variables like 'max_connections';
show full processlist;