Understanding FPGA: Architecture, Advantages, and Microsoft’s Data‑Center Deployments

FPGA (Field‑Programmable Gate Array) is a reconfigurable hardware architecture that differs fundamentally from the von Neumann designs of CPUs and GPUs because it has no instruction decoder and does not rely on shared memory, giving it higher energy efficiency and lower latency.

Compared with CPUs, GPUs, and ASICs, FPGA provides a unique blend of pipeline parallelism and data parallelism, allowing fine‑grained, stream‑oriented processing with microsecond‑level PCIe latency. This makes FPGA especially suitable for latency‑sensitive, streaming workloads such as search ranking, encryption, and network packet processing.

Microsoft has deployed FPGA in its data centers through three evolutionary stages: (1) dedicated FPGA clusters, (2) one FPGA per server connected by a custom network, and (3) integrating FPGA between the NIC and the switch (SmartNIC) to accelerate network and storage virtualization. The third‑generation architecture uses a lightweight transport layer (LTL) to interconnect thousands of FPGAs across the data center with sub‑10 µs latency.

Performance measurements show that FPGA can match or exceed CPU/GPU throughput for integer and floating‑point operations while delivering orders‑of‑magnitude lower and more stable latency for packet processing. In Bing’s search ranking pipeline, a 1632‑node FPGA cluster doubled performance and halved server count.

Programming FPGA as a compute accelerator using OpenCL incurs unnecessary DRAM round‑trips; instead, Microsoft’s ClickNP framework uses channel‑based communication (CSP model) between kernels and between FPGA and host, achieving microsecond‑level data transfer and eliminating the shared‑memory bottleneck.

The article argues that FPGA should be viewed as a complementary “large‑scale network‑accelerator” in cloud environments, handling repetitive, locality‑rich tasks (network, storage, encryption, DNN inference) while CPUs handle complex, irregular workloads, enabling a scalable, flexible, and efficient heterogeneous cloud architecture.