How to Detect and Investigate Linux Intrusions: Essential Commands and Techniques

Account Security

Inspect user information files such as /etc/passwd (format: account:password:UID:GID:GECOS:directory:shell) and /etc/shadow (encrypted passwords and expiration details). Use commands like cat /etc/passwd | grep /bin/bash, awk -F: '$3==0{print $1}' /etc/passwd, and more /etc/sudoers | grep -v "^#\|^$" | grep "ALL=(ALL)" to view login‑capable users and sudo privileges.

Note: Accounts without passwords can only log in locally.

Login Activity Inspection

Check current logged‑in users and session duration with who, w, and uptime. Use last to display recent successful logins and lastb for failed attempts. Protect /var/log/wtmp from deletion by setting the immutable attribute: chattr +a /var/log/wtmp.

Privilege and Sudo Users

List sudoers via /etc/sudoers. Identify privileged (UID 0) users with awk -F: '$3==0{print $1}' /etc/passwd and remote‑login capable accounts with awk '/\$1|\$6/{print $1}' /etc/shadow. Disable or delete unnecessary accounts using usermod -L user or userdel user.

Command History Analysis

Examine each user’s .bash_history for executed commands. Extend history size and include IP and timestamps by editing /etc/profile:

# Increase history size
sed -i 's/^HISTSIZE=1000/HISTSIZE=10000/g' /etc/profile
# Append IP and user info
USER_IP=`who -u am i 2>/dev/null | awk '{print $NF}' | sed -e 's/[()]//g'`
if [ "$USER_IP" = "" ]; then USER_IP=`hostname`; fi
export HISTTIMEFORMAT="%F %T $USER_IP `whoami` "
shopt -s histappend
export PROMPT_COMMAND="history -a"
source /etc/profile

Clear history with history -c and remove the saved file .bash_profile if needed.

Port and Process Checks

List listening ports: netstat -antlp | more. Identify processes by PID: ps aux | grep 6666, then locate the executable with ls -l /proc/$PID/exe or file /proc/$PID/exe. Sort processes by CPU or memory usage using ps -ef --sort -pcpu and ps -ef --sort -pmem. Terminate suspicious processes with kill -9 $PID.

Startup Script Inspection

Review runlevels (0‑6) and associated scripts in /etc/rc.local, /etc/rc.d/rc[0~6].d, or via runlevel. Add custom scripts to /etc/rc.local before the exit 0 line, or use update-rc.d to create symbolic links in /etc/init.d and the appropriate /etc/rc*.d directories.

Cron Job Auditing

List user crontabs with crontab -l and remove them with crontab -r. System‑wide cron files reside in /etc/crontab, /etc/cron.d/, and the hourly/daily/weekly/monthly directories. Examine them using more /etc/cron.daily/* and search for suspicious entries.

File Search Techniques

Find files by name, size, modification time, or ownership:

# By name (wildcards allowed)
find / -name "a.Test"
# Larger than 1000M
find / -size +1000M
# Modified within the last day
find / -mtime -1 -ls | more
# Owned by root
find . -user root -type f

Log File Examination

Key logs are located under /var/log/, especially secure and history. Important log files include /var/log/cron, /var/log/message, /var/log/secure, /var/log/wtmp, /var/log/lastlog, and /var/log/btmp. Use grep and awk to extract failed‑password attempts, successful logins, user additions, deletions, and sudo usage.

WebShell Removal and Security Scripts

Reference external tools for webshell detection (e.g., shellpub.com ) and open‑source security scanners such as https://github.com/grayddq/GScan, https://github.com/ppabc/security_check, and https://github.com/T0xst/linux.