Escaping VMware: The Story Behind a VM Escape Exploit and Anti‑VM Tricks

Falling into a Trap

Two intruders enter a new system at night. The "leader" plans to scan the disk while the "assistant" modifies the boot startup items. Suddenly they notice the environment is unusually quiet—no QQ or WeChat processes appear. The leader then realizes they are inside a virtual machine after spotting a VMware process and registry markers.

Realizing they are trapped, the leader produces a secret note left by the host that describes a "VM escape method". The note lists several CVE identifiers:

CVE-2016-7461 CVE-2017-4901 CVE-2019-14378

The assistant asks what the numbers mean. The leader explains that Common Vulnerabilities and Exposures (CVE) is a public identifier where the year follows the prefix and the remaining digits specify the particular vulnerability.

Choosing CVE‑2017‑4901, which affects VMware, they learn that VMware provides a backdoor communication interface used for copy‑and‑paste and drag‑and‑drop between the guest and host. This backdoor contains a heap‑overflow bug; by crafting malicious data, they can trigger the overflow and execute arbitrary code inside the host.

New World

Following the instructions in the note, the leader prepares the exploit code and runs it. The exploit succeeds, the two characters’ bit‑stream is transferred into the host’s VMware process, and they find themselves in the host’s file system. They celebrate their escape, but soon notice that a VMware process is still running.

Investigating further, they discover that the environment they have entered is itself another virtual machine, meaning they are still trapped inside a nested VM.

Story Inspiration

The story was inspired by a question on Zhihu: "Can an operating system know it is running inside a virtual machine?" The author’s answer outlines three stages of virtualization technology:

Early binary translation (e.g., early VMware) OS‑level hypervisors that modify special instruction calls (e.g., Xen) Hardware‑assisted virtualization (VT‑x), which sparked a boom in solutions such as VirtualBox, modern VMware, and KVM.

Even with advanced hardware virtualization, detecting a VM remains straightforward. Malware authors often employ anti‑VM techniques, checking for special I/O devices, registry keys, process names, or file system artifacts. More sophisticated methods involve side‑channel analysis, such as measuring execution timing or hardware state changes.

Virtual machines are not absolutely secure; they share the physical CPU with the host, and any flaw in the isolation mechanisms can allow malicious code to escape, as illustrated by the story’s VM‑escape exploit.