How to Diagnose and Remove a Linux Backdoor That Triggers Massive Outbound Traffic

1. Problem Investigation

The monitoring system sent repeated "ping unreachable" alerts, and Zabbix showed outbound traffic spiking to about 800 MB, which was abnormal. SSH access failed due to network congestion.

Initial reaction was to ask the data‑center staff to cut the external network, but that would hide the attack source.

Instead, the data‑center team logged in locally and examined the w command output and /var/log/auth.log, which was already cleared. The iftop tool revealed continuous HTTP traffic to 104.31.225.6. The IP was blocked with: iptables -A OUTPUT -d 104.31.225.6 -j DROP Traffic dropped instantly and SSH became usable again, but the spike returned shortly after, this time from a different IP.

2. Locating the Attack Source

Using netstat -atup | grep 15773 showed no listening process on the suspected port. The connection was short‑lived, so it disappeared from the netstat view.

For long‑lived connections, lsof -i :15773 can reveal the PID, then lsof -p PID shows the associated files.

Because no suspicious ports or processes were found, the team decided to block external network again, SSH in, and search for the program generating the traffic.

Typical checks included: netstat -antup – look for unusual open ports or connections. ps -ef – look for suspicious processes.

No obvious anomalies appeared, raising the suspicion of a rootkit.

To verify binary integrity, the author considered comparing md5sum of system tools with those from a clean OS, but the exact OS version was unavailable, making the check unreliable.

Running du -sh /bin/lsof showed an unexpectedly large size (1.2 MB), indicating a tampered binary.

Clean versions of netstat, ps, etc., were copied from a healthy system and replaced the compromised ones, after which the traffic stopped.

3. Cleaning the Malware

After executing ps -ef, several suspicious entries appeared:

/sbin/java.log

/usr/bin/dpkgd/ps -ef

/usr/bin/bsd-port/getty

/usr/bin/.sshd

Each of these binaries resided in non‑standard directories, suggesting they had been replaced by the attacker.

The java.log process kept restarting, indicating an auto‑start script in /etc/init.d. Two malicious scripts were found there and deleted, along with the java.log file, stopping the process.

All identified malicious processes were terminated and their files removed. After cleanup, the external network was re‑enabled and traffic remained normal.

4. Summary and Recommendations

Key artifacts of the compromise: /usr/bin/dpkgd/ – directory containing replaced tools (netstat, lsof, ps, ss). /sbin/java.log – backdoor program that regenerated itself. /usr/bin/bsd-port – used to generate java.log or other backdoor binaries. /usr/sbin/.sshd – hidden SSH backdoor.

Preventive measures recommended:

Enable a host firewall and allow only trusted sources to access required services.

Collect and monitor system, login, and application logs; set alerts for abnormal keywords.

Record user login details, password‑retry counts, and command execution history.

Monitor critical files and directories (e.g., /etc/passwd, /etc/shadow, /web, /tmp) for changes.

Continuously monitor process creation and notify on unknown processes.

Regularly scan servers and web applications for known vulnerabilities.

In cases where the system cannot be reinstalled immediately, a phased data migration and continued observation are advised, but a full reinstall remains the cleanest remediation.