How I Detected and Eradicated a Linux Rootkit on My Home Network

One evening, while trying to call a friend, the author noticed QQ going offline and back online automatically, suspecting a hack. After the call, the issue was traced to an unstable broadband connection, but soon even basic sites like Baidu became unreachable.

Event Origin

The author logged into the router and found a Linux virtual machine monopolizing all upstream traffic, indicating a possible malware infection.

Investigation Steps

1. Examine processes with ps and top

Two processes with random English names appeared in ps. The top screenshot showed high bandwidth usage.

Killing these processes temporarily reduced bandwidth, but they reappeared quickly, suggesting a background daemon.

2. Disable the VM network card

The VM NIC was switched to vmnet host only mode and its IP set to the same subnet as the VMware virtual NIC to allow SSH access while isolating it from the internet.

3. Locate the program files

Using ls /proc/\<pid\>/exe revealed that the malicious binaries changed paths within the PATH directories, appearing in /bin, /sbin, and /usr/bin.

4. Search for hidden files

Scanning /bin, /sbin, and /usr/bin for files starting with a dot uncovered numerous hidden executables that regenerated after deletion.

Because strace was not installed, the author mounted an ISO, created a local YUM repository, and installed it.

Running strace /bin/hzqzqdmatu showed the process self‑deleting its executable file.

Further checks with netstat revealed no external connections, and timestamps of common commands ( ps, ls, netstat, etc.) indicated they had been modified on June 24, suggesting a user‑mode rootkit.

The author also inspected for additional admin accounts; only root existed, confirming no extra users were created.

5. Delete malicious binaries and reboot

All identified malicious programs were forcefully removed, the VM was powered off, and then rebooted.

Despite the reboot, the programs reappeared, indicating they were registered as startup services across multiple run levels.

After another round of forced deletion and reboot, CPU usage dropped and the malicious processes ceased.

The author concluded that once a rootkit compromises the system, the only reliable remedy is a full reinstall. For demonstration, a recovery method on CentOS 6.8 is described: identify the RPM packages providing compromised commands, force‑remove them, and reinstall via a local YUM source.

Finally, the author emphasizes that systematic troubleshooting—examining processes, network activity, and system logs—is essential, and encourages continuous learning to handle such security incidents effectively.