Incident Report: Investigating and Removing a Server Malware Causing 100% CPU Usage
This article documents a step‑by‑step investigation of a compromised Linux server that exhibited 100% CPU usage, detailing process, network, and startup‑service analysis, the discovery of a cryptomining malware, and the complete removal procedure.
This document records the handling process of a server that was compromised, providing detailed steps and a summary for readers who face similar issues.
Background : A developer reported that after deployment the application could not run and kept restarting. Service logs showed no errors, but monitoring revealed CPU utilization at 100% on several nodes.
Process inspection : Using the top command showed the CPU fully saturated, yet no abnormal process appeared; stopping a few business processes did not improve the situation.
Network inspection : Suspecting intrusion, the command netstat -an | grep ESTABLISHED was executed. All affected machines displayed an external connection on port 7643 to foreign IP addresses, indicating malicious activity.
Startup items : The command systemctl list-unit-files | grep enabled revealed a suspicious service named OOlmeN2R.service . The service list included:
auditd.service enabled
[email protected] enabled
crond.service enabled
docker.service enabled
OOlmeN2R.service enabled <-------
... (other services omitted) ...
sshd.service enabledInspecting the service with systemctl status OOlmeN2R.service and viewing its unit file ( /usr/lib/systemd/system/OOlmeN2R.service ) showed:
[Unit]
Description=service
After=network.target
[Service]
Type=simple
ExecStart=/bin/eWqAVtbn
RemainAfterExit=yes
Restart=always
RestartSec=60sThe ExecStart points to /bin/eWqAVtbn , which is the malware executable.
Virus removal :
Stop and disable the service: systemctl stop OOlmeN2R.service systemctl disable OOlmeN2R.service
Delete the malicious files: rm /bin/eWqAVtbn # delete executable rm /usr/lib/systemd/system/OOlmeN2R.service # delete unit file
Reboot the server.
After reboot, running netstat -an | grep ESTABLISHED showed no suspicious connections and CPU usage returned to normal, confirming the malware was removed.
Summary : The malware is likely a cryptomining trojan that hides its process, persists via startup items, uses random filenames, and consumes massive CPU resources. The documented investigation and cleanup steps effectively eliminated the infection.
DevOps Operations Practice
We share professional insights on cloud-native, DevOps & operations, Kubernetes, observability & monitoring, and Linux systems.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.