7 Practical Linux Performance Optimization Techniques Every Sysadmin Should Know

1. Factors that Influence Linux System Performance

CPU load and utilization

Memory usage and availability

Disk I/O throughput and latency

Network traffic and latency

Process scheduling policies

Filesystem choice and configuration

Kernel parameters (e.g., TCP/IP, memory management, filesystem cache)

Resource limits and quotas in multi‑user environments

Optimizing Linux performance requires a holistic review of these interrelated components.

2. Quick Fault‑Isolation Techniques

CPU analysis : Use top, vmstat, pidstat, strace or perf to capture CPU metrics, then correlate with process behavior to locate bottlenecks.

Memory analysis : Examine free and vmstat output, identify high‑memory processes, and investigate possible leaks or excessive caching.

Disk & filesystem I/O analysis : Run iostat to spot I/O saturation, then drill down with pidstat or vmstat and inspect the offending processes using strace, lsof or perf.

Network analysis : Check interface throughput, packet loss, errors, and soft‑IRQ counts; examine routing, fragmentation, and TCP/UDP statistics; use netstat, tcpdump or bcc to pinpoint the layer where the problem originates.

3. Diagnosing High System Load

Run top or htop and note the load‑average; values above 70‑80 % of CPU cores indicate overload.

Identify processes with high CPU usage via top.

Check memory pressure with free; watch for swapping.

Inspect disk I/O using iotop.

Review active network connections with netstat or similar tools.

Search system and application logs for errors.

Employ perf or strace for detailed per‑process profiling.

Verify kernel and sysctl settings (e.g., /etc/sysctl.conf) match workload requirements.

Following these steps helps isolate the root cause of load spikes.

4. Finding the Top‑5 Resource‑Consuming Processes

CPU ranking: ps aux --sort=-%cpu | head -n 5 Memory ranking: ps aux --sort=-%mem | head -n 6 I/O ranking (interactive view): iotop -oP Alternative commands:

# ps aux | grep -v USER | sort +2 | tail -n 10   # top CPU

# ps aux | grep -v USER | sort +3 | tail -n 10   # top memory

iostat 1 10   # identify busy disks

5. Why Linux Memory Statistics May Appear Inaccurate

Differences arise because free reports instantaneous counts while /proc/meminfo reflects real‑time changes, and because free includes cache and buffer memory as used. Additional factors:

Cache & buffer memory is reserved by the kernel but not owned by processes.

Shared memory segments are counted differently across tools.

Kernel memory reclamation may delay visible changes.

For precise per‑process memory usage, tools such as pmap or smem are recommended.

6. Current Use Cases for Swap

Memory shortage: provides overflow space when RAM is exhausted.

Hibernate/suspend: stores RAM contents to disk for state restoration.

Virtualization: allows host to swap out guest pages when physical memory is scarce.

Memory reclamation & page replacement: frees RAM for higher‑priority workloads.

Excessive swapping degrades performance; proper RAM sizing is advised.

7. Linux TCP Tuning Experience

Scenario 1 – High connection volume (many TIME_WAIT sockets)

Increase net.ipv4.tcp_max_tw_buckets and net.netfilter.nf_conntrack_max.

Reduce net.ipv4.tcp_fin_timeout and net.netfilter.nf_conntrack_tcp_timeout_time_wait.

Enable port reuse with net.ipv4.tcp_tw_reuse.

Expand local port range via net.ipv4.ip_local_port_range.

Raise file‑descriptor limits using fs.nr_open, fs.file-max or LimitNOFILE in systemd.

Scenario 2 – SYN‑FLOOD or half‑open connection attacks

Raise net.ipv4.tcp_max_syn_backlog or enable SYN cookies with net.ipv4.tcp_syncookies (mutually exclusive).

Decrease retransmission attempts with net.ipv4.tcp_synack_retries.

Scenario 3 – Long‑lived connections (keepalive tuning)

Shorten keepalive interval: net.ipv4.tcp_keepalive_time.

Reduce probe spacing: net.ipv4.tcp_keepalive_intvl.

Limit retry count: net.ipv4.tcp_keepalive_probes.

Applying these kernel parameters improves TCP throughput, reduces resource consumption, and mitigates common denial‑of‑service vectors.