How to Diagnose and Eradicate a Linux Trojan That Spikes Outbound Traffic

Background

A remote Ubuntu 12.04 (x64) server hosting business applications, crawlers and data queues began generating ~800 MB of outbound traffic, making SSH connections unreliable.

Initial Investigation

Local console access was obtained via the data‑center staff. The authentication log (/var/log/auth.log) was empty, so iftop was used to monitor network usage. Continuous HTTP requests to 104.31.225.6 were observed. An iptables rule was added to drop traffic to that address: iptables -A OUTPUT -d 104.31.225.6 -j DROP The traffic stopped briefly but resumed from a different IP, indicating a locally running program was generating the packets.

Locating the Malicious Process

Standard network utilities were employed: netstat -atup | grep 15773 – no persistent connections were found.

Because the connections were short‑lived, lsof -i :15773 and lsof -p <PID> were suggested for deeper inspection.

System binaries appeared tampered (e.g., netstat, ps, lsof, ss). Clean versions of these utilities were copied from a trusted Ubuntu installation and uploaded to the compromised host for reliable analysis.

Identifying and Removing the Trojan

Running ps -ef revealed several suspicious processes and binaries that do not exist in a standard Ubuntu installation:

/sbin/java.log

/usr/bin/dpkgd/ps –ef

/usr/bin/bsd-port/getty

/usr/bin/.sshd

Each process was terminated and the corresponding files deleted. The .sshd binary behaved as a backdoor, while java.log re‑appeared after removal, indicating an auto‑start script in /etc/init.d. Deleting that script stopped the recurring launch.

Further inspection showed a directory /usr/bin/dpkgd/ that contained replaced utilities (e.g., netstat, lsof, ps, ss) which were either non‑functional or of abnormal size (e.g., lsof reported 1.2 MiB).

Key Artifacts

ls /usr/bin/dpkgd/

– directory holding malicious replacements for standard tools. /sbin/java.log – the primary outbound program that regenerated automatically. /usr/bin/bsd-port – script used to launch java.log or other backdoor components. /usr/sbin/.sshd – backdoor executable masquerading as SSH.

The trojan source was shared via a Baidu link (http://pan.baidu.com/s/1b3yOVW). Executing java.log can render the host unreachable.

Preventive Measures

To mitigate similar incidents, the following hardening steps are recommended:

Enable a host‑based firewall (e.g., iptables or ufw) that only permits trusted sources to access required services.

Remove unnecessary user accounts and disable unused services.

Collect and continuously monitor system, authentication, and application logs for anomalies.

Track login attempts, password retries, and command execution per user.

Watch critical files and directories (e.g., /etc/passwd, /etc/shadow, web roots, /tmp) for unexpected changes.

Record and alert on creation of new or suspicious processes.

Regularly scan servers and web applications for known vulnerabilities.

Reducing the attack surface and improving visibility are essential for maintaining a resilient Linux environment.