How Does a Host Discover and Access PCIe Devices? A Step‑by‑Step Walkthrough

PCIe (Peripheral Component Interconnect Express) is the modern high‑speed interface used by SSDs, NICs and other peripherals, built on a hierarchical topology of root complexes, switches and endpoints. Each device is identified by a BDF (Bus:Device:Function) address, allowing up to 256 buses, 32 devices per bus, and 8 functions per device.

Host Enumeration Process

The host discovers PCIe devices using a depth‑first search algorithm, commonly called PCIe device enumeration. The process visits each possible branch until no further devices are found, ensuring each node is accessed only once.

Step 1 – Scan Bus 0: The host bridge scans Bus 0, ignores embedded endpoints, discovers Bridge 1, assigns it Bus 1, and initializes its configuration space. Primary and Secondary Bus Number registers are set to 0 and 1, while Subordinate Bus Number is temporarily 0xFF.

Step 2 – Scan Bus 1: Bridge 1 reveals Bridge 3 (a switch). Bridge 3 receives Bus 2, with Primary/Secondary numbers 1/2 and Subordinate set to 0xFF.

Step 3 – Scan Bus 2: Bridge 3 exposes Bridge 4, which in turn connects to an NVMe SSD endpoint. Bridge 4 gets Bus 3; its Subordinate Bus Number becomes 3 because Bus 3 hosts only leaf devices.

Step 4 – Scan Bus 3: After completing Bus 3, the host returns to Bus 2, discovers Bridge 5, which connects a NIC endpoint. Bridge 5 receives Bus 4, and its Subordinate Bus Number is set to 4.

Step 5 – Finalize Subordinate Numbers: With no more devices under Bus 2, Bridge 3’s Subordinate Bus Number is set to 4. The host then returns to Bridge 1, also setting its Subordinate Bus Number to 4.

Step 6 – Scan Remaining Bus 0 Devices: The host discovers Bridge 2, assigns it Bus 5, and records the graphics card as an endpoint, finalizing Bridge 2’s Subordinate Bus Number as 5.

After these steps, the host has built a complete PCIe topology tree, which remains static after boot; hot‑plug devices are not rescanned in typical systems.

Inspecting Devices on Linux

Linux provides the lspci -v -t command to display the PCIe tree. For example, the output shows a STAR1000 NVMe SSD controller from Beijing Starblaze Technology with BDF 3c:00.0 (bus 0x3C, device 0x00, function 0x0) and upstream port 00:1d.0.

Detailed configuration space can be examined with lspci -xxx -s 3c:00.0, revealing Vendor ID, Device ID, class code 0x010802 (NVMe storage), and a series of capability structures starting at offset 0x40 (power management, MSI, link control, etc.). The link capability field 0x43 indicates a x4 lane, Gen3 (8 Gbps) link.

Alternatively, lspci -vvv -s 3c:00.0 provides a verbose view of all capabilities.

PCIe Memory Access and NVMe Commands

Each PCIe device is allocated one or more CPU memory windows. The STAR1000 controller exposes a 1 MiB window and a 256 KiB window, each with a base address programmed by the host. The host accesses device registers and queues through these memory regions.

NVMe command submission works via memory writes: the host writes to the doorbell register (a memory‑write request) to notify the SSD of a new command. The SSD then issues a memory‑read request to fetch the command, processes it, and the host receives a completion packet (CplD) via a memory‑read response.

All other NVMe operations—queue management, data transfer, status reporting—are performed through similar PCIe memory accesses.

Conclusion

The enumeration algorithm builds a full PCIe device map, and the memory‑mapped I/O mechanism enables efficient command exchange between host and NVMe devices. Understanding these steps is essential for debugging, performance tuning, and designing systems that leverage high‑speed PCIe storage.