Do CPUs Hide Secret Instructions? Uncovering the Dark Corners of the x86 ISA

Instruction Set Search Space

The x86 instruction set is a complex CISC architecture where instructions vary from one to many bytes. Single‑byte opcodes (e.g., 90 for nop , CC for int 3 , C3 for ret ) are listed in Intel manuals, but many opcode slots are empty, such as 0xF1. The report asks whether these gaps hide undocumented instructions.

Instruction Search Algorithm

The authors propose a depth‑first search that quickly skips bytes deemed irrelevant. For example, all push instructions start with the opcode 68 followed by four data bytes; only the first byte matters for control‑flow, so the algorithm treats the remaining bytes as ignorable. By recognizing such patterns, the search space shrinks dramatically.

68 6F 72 6C 64   push 646C726Fh 68 6F 2C 20 77   push 77202C6Fh 68 68 65 6C 6C   push 6C6C6568h

The algorithm iteratively increments the last byte of a candidate instruction, executes it, and observes whether the CPU raises a page‑fault exception (indicating the instruction crossed into a non‑executable page). If the exception address stays on the non‑executable page, the instruction length is longer than the bytes examined, prompting further exploration.

Determining Instruction Length

To decide if a modified byte affects instruction length, the method places a sequence of bytes across two memory pages: the first page is executable, the second is not. By jumping to the sequence and watching where a fault occurs, the algorithm infers the exact boundary of the instruction. Repeating this with increasing numbers of bytes (1, 2, 3, …) eventually yields the minimal length that executes without fault, revealing the true instruction size.

Mining Hidden Instructions

Applying the technique to an Intel Core i7 CPU uncovered numerous opcodes that are not documented in the official manual but execute on real hardware. Similar experiments on AMD Athlon CPUs produced additional hidden instructions. Some of these have been reverse‑engineered, while others remain mysterious, known only to the silicon vendors.

These findings highlight that CPUs can contain undocumented functionality, which may be exploited for malicious purposes or, conversely, represent a security risk that is difficult to audit.

Source report: us-17-Domas-Breaking-The-x86-ISA by @xoreaxeaxeax.