10 Common Linux Ops Problems and Proven Fixes

Linux Operations Practical Guide: 10 Common Issues and Solutions

As a senior operations engineer, I have compiled the most frequently encountered Linux system problems in daily work. These cover system performance, disk management, network configuration, and more, each with detailed diagnostic steps and multiple solutions.

1. High System Load and Slow Response

Symptoms

System response time noticeably increases

User interactions become sluggish

Applications start slowly

Diagnostic Steps

# Check system load
uptime
top -c
htop

# Check CPU usage
vmstat 1 5
iostat -x 1 5

# Check memory usage
free -h
cat /proc/meminfo

Solution

Temporary Fix:

# Find the process using the most CPU
ps aux --sort=-%cpu | head -10

# Kill abnormal processes (use with caution)
kill -9 PID

# Clear system cache (use with caution)
echo 3 > /proc/sys/vm/drop_caches

Long‑Term Optimization:

Adjust process priority: nice -n 10 command Tune system parameters: edit /etc/sysctl.conf Upgrade hardware resources or optimize applications

2. Disk Space Exhaustion

Symptoms

System reports "No space left on device"

Unable to create new files

Applications crash unexpectedly

Diagnostic Steps

# Check disk usage
df -h
du -sh /*
du -sh /var/log/*

# Find large files
find / -type f -size +100M -exec ls -lh {} \;
find /var/log -name "*.log" -size +50M

# Check inode usage
df -i

Solution

Immediate Cleanup:

# Clean system logs
journalctl --vacuum-time=7d
logrotate -f /etc/logrotate.conf

# Remove temporary files
rm -rf /tmp/*
rm -rf /var/tmp/*

# Clean package cache
apt clean   # Ubuntu/Debian
yum clean all   # CentOS/RHEL

Preventive Measures:

# Set up log rotation
cat > /etc/logrotate.d/custom <<EOF
/var/log/application.log {
    daily
    rotate 7
    compress
    delaycompress
    missingok
    notifempty
    create 644 user group
}
EOF

# Add disk monitoring to cron
echo "*/10 * * * * root df -h | awk '$5 > 80 {print $0}' | mail -s 'Disk Usage Alert' [email protected]" >> /etc/crontab

3. Network Connectivity Issues

Symptoms

Cannot access external network

Inter‑service communication fails

High network latency

Diagnostic Steps

# Check network interfaces
ip addr show
ifconfig

# Test connectivity
ping -c 4 8.8.8.8
traceroute google.com
mtr google.com

# Examine routing table
ip route show
route -n

# Check DNS resolution
nslookup google.com
dig google.com

# Inspect firewall status
iptables -L -n
firewall-cmd --list-all

Solution

Network Configuration Repair:

# Restart networking service
systemctl restart networking   # Ubuntu
systemctl restart network      # CentOS

# Manually set IP (temporary)
ip addr add 192.168.1.100/24 dev eth0
ip route add default via 192.168.1.1

# Update DNS configuration
echo "nameserver 8.8.8.8" > /etc/resolv.conf
echo "nameserver 8.8.4.4" >> /etc/resolv.conf

Firewall Configuration:

# Allow common ports
iptables -A INPUT -p tcp --dport 80 -j ACCEPT
iptables -A INPUT -p tcp --dport 443 -j ACCEPT

# Save firewall rules
iptables-save > /etc/iptables/rules.v4

4. Service Fails to Start

Symptoms

"systemctl start" command fails

Service status shows "failed"

Application ports not listening

Diagnostic Steps

# Check service status
systemctl status service_name
journalctl -u service_name -n 50

# Inspect configuration files
systemctl cat service_name
nginx -t   # Nginx
apache2ctl configtest   # Apache

# Check port usage
netstat -tulpn | grep :80
ss -tulpn | grep :80
lsof -i :80

Solution

Configuration File Repair:

# Backup original config
cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak

# Test syntax and reload
nginx -t
systemctl reload nginx

# List service dependencies
systemctl list-dependencies service_name

Permission Issues:

# Check file permissions
ls -la /var/log/nginx/
chown -R nginx:nginx /var/log/nginx/
chmod 755 /var/log/nginx/

# Check SELinux status
getenforce
setsebool -P httpd_can_network_connect 1

5. SSH Connection Refused

Symptoms

"Connection refused" error

"Permission denied" prompt

Connection timeout

Diagnostic Steps

# Check SSH service status
systemctl status sshd
ps aux | grep sshd

# Review SSH configuration
sshd -T | grep -E "(port|permitrootlogin|passwordauthentication)"

# Verify network listening
netstat -tulpn | grep :22
iptables -L | grep ssh

# Examine authentication logs
tail -f /var/log/auth.log
journalctl -u sshd -f

Solution

Service Repair:

# Restart SSH service
systemctl restart sshd

# Test configuration syntax
sshd -t

# Change SSH port if needed
sed -i 's/#Port 22/Port 2222/' /etc/ssh/sshd_config
systemctl reload sshd

Security Hardening:

# Disable root login
sed -i 's/#PermitRootLogin yes/PermitRootLogin no/' /etc/ssh/sshd_config

# Enable key authentication
sed -i 's/#PubkeyAuthentication yes/PubkeyAuthentication yes/' /etc/ssh/sshd_config

# Limit login attempts
echo "MaxAuthTries 3" >> /etc/ssh/sshd_config
echo "MaxStartups 10:30:60" >> /etc/ssh/sshd_config

6. Excessive Memory Usage

Symptoms

System becomes sluggish

Applications are killed by OOM killer

Swap usage is high

Diagnostic Steps

# View memory details
free -h
cat /proc/meminfo
vmstat 1 5

# Identify memory‑hungry processes
ps aux --sort=-%mem | head -10
top -o %MEM

# Check swap usage
swapon -s
cat /proc/swaps

# Look for OOM killer logs
dmesg | grep -i "killed process"
journalctl -k | grep -i "killed process"

Solution

Temporary Memory Release:

# Drop caches
echo 3 > /proc/sys/vm/drop_caches
sync

# Restart high‑memory services
systemctl restart high_memory_service

# Adjust swap aggressiveness
echo 10 > /proc/sys/vm/swappiness

Long‑Term Optimization:

# Permanently set swappiness
echo "vm.swappiness=10" >> /etc/sysctl.conf

# Set memory limits in systemd services
[Service]
MemoryLimit=1G
MemoryMax=1G

7. Filesystem Errors

Symptoms

Filesystem mounted read‑only

Files corrupted or missing

Disk I/O errors

Diagnostic Steps

# Check filesystem status
mount | grep "ro,"
df -h

# Look for disk errors
dmesg | grep -i error
cat /var/log/messages | grep -i error

# Examine disk health
smartctl -a /dev/sda
badblocks -v /dev/sda1

Solution

Filesystem Repair:

# Unmount if possible
umount /dev/sda1

# Run filesystem checks
fsck -f /dev/sda1
 e2fsck -f /dev/sda1   # ext filesystems
 xfs_repair /dev/sda1   # XFS

# Remount read‑write
mount -o remount,rw /

Preventive Measures:

# Schedule regular checks
echo "0 2 * * 0 root fsck -A -R -T -C -a" >> /etc/crontab

# Monitor disk health
smartctl -t short /dev/sda
smartctl -a /dev/sda

8. Time Synchronization Problems

Symptoms

System clock is inaccurate

Log timestamps are chaotic

Authentication failures

Diagnostic Steps

# View current time
date
timedatectl status

# Check NTP services
systemctl status ntp
systemctl status chrony
ntpq -p

# Verify timezone settings
ls -la /etc/localtime
cat /etc/timezone

Solution

NTP Configuration:

# Install NTP
apt install ntp   # Ubuntu
yum install ntp   # CentOS

# Configure NTP servers
cat > /etc/ntp.conf <<EOF
server 0.pool.ntp.org iburst
server 1.pool.ntp.org iburst
server 2.pool.ntp.org iburst
EOF

# Enable and start NTP
systemctl enable ntp
systemctl start ntp

Using timedatectl (recommended):

# Enable NTP
timedatectl set-ntp true

# Set timezone
timedatectl set-timezone Asia/Shanghai

# Manual sync if needed
ntpdate -s time.nist.gov

9. Zombie/Orphan Processes

Symptoms

Large number of zombie processes

Processes cannot be terminated

Resources are not released

Diagnostic Steps

# List zombie processes
ps aux | awk '$8 ~ /^Z/ {print $0}'
ps -eo pid,stat,comm | grep Z

# View process tree
pstree -p
ps -ef --forest

# Inspect specific process status
cat /proc/PID/status
ls -la /proc/PID/

Solution

Clean Zombie Processes:

# Identify parent and send SIGCHLD
ps -o pid,ppid,state,comm | grep Z
kill -CHLD parent_pid

# Force‑kill process groups if needed
kill -9 -process_group_id

# Restart affected services
systemctl restart problematic_service

Preventive Measures:

# Script to monitor zombies
cat > /usr/local/bin/zombie_monitor.sh <<'EOF'
#!/bin/bash
zombies=$(ps aux | awk '$8 ~ /^Z/ {print $2}' | wc -l)
if [ $zombies -gt 10 ]; then
    echo "Detected $zombies zombie processes" | mail -s "Zombie Process Alert" [email protected]
fi
EOF
chmod +x /usr/local/bin/zombie_monitor.sh
echo "*/5 * * * * root /usr/local/bin/zombie_monitor.sh" >> /etc/crontab

10. System Security Issues

Symptoms

Abnormal network connections

Unknown processes running

System files have been modified

Diagnostic Steps

# Check suspicious connections
netstat -antup | grep ESTABLISHED
ss -tuln

# Review login records
last -n 20
lastlog
who -a

# Find suspicious processes
ps aux | grep -v "\["
lsof -i
find /tmp -type f -executable

# Verify system integrity
rpm -Va   # RHEL/CentOS
debsums -c   # Ubuntu/Debian

Solution

Security Hardening:

# Apply system updates
apt update && apt upgrade   # Ubuntu
yum update                  # CentOS

# Configure firewall
ufw enable                  # Ubuntu
firewall-cmd --permanent --add-service=ssh
firewall-cmd --reload       # CentOS

# Install security tools
apt install fail2ban rkhunter chkrootkit

Monitoring Script:

# Create security check script
cat > /usr/local/bin/security_check.sh <<'EOF'
#!/bin/bash
# Detect abnormal logins
lastlog | awk '$2 !~ /Never/ && $2 !~ /pts/ {print "Abnormal login: " $0}'

# Detect suspicious processes
ps aux | awk '$11 ~ /^\[/ {next} $1 == "root" && $11 !~ /^\// {print "Suspicious process: " $0}'

# Detect external network connections
netstat -antup | awk '$6 == "ESTABLISHED" && $5 !~ /^(127\.|192\.168\.|10\.)/ {print "External connection: " $0}'
EOF
chmod +x /usr/local/bin/security_check.sh
echo "0 */6 * * * root /usr/local/bin/security_check.sh | mail -s 'Security Check Report' [email protected]" >> /etc/crontab

Conclusion

These ten problems cover the core scenarios encountered in Linux operations. As an operations engineer, mastering systematic diagnosis methods and solution thinking is more important than memorizing individual commands.

Best‑Practice Recommendations:

Establish Monitoring Systems: Use Prometheus + Grafana, Zabbix, etc.

Automate Operations: Write scripts to handle common issues.

Regular Backups: Backup system configurations and critical data.

Documentation Management: Record detailed steps for each incident.

Prevention Over Cure: Leverage monitoring and alerts to avoid problems.