The Legendary Intel FDIV Bug: How a Hidden Flaw Shook the Computing World

Background

Intel’s Pentium processor implemented the floating‑point division instruction FDIV with a high‑performance variant of the Sweeney‑Robertson‑Tocher (SRT) algorithm. The implementation relied on a 2048‑entry hardware lookup table, of which only 1066 entries are ever accessed. Five of those accessed entries were erroneously programmed to 0 instead of the correct value 2. Because the faulty entries are reached only for a very narrow set of divisor patterns, the error manifests roughly once every 9 billion division operations (≈ one occurrence per 700 years of continuous use at typical workloads).

Detection

Professor Thomas Nicely (Lynchburg College) was computing the sum of reciprocals of twin primes on several machines. He observed that the results differed between systems. After months of investigation he isolated the cause to machines equipped with the Pentium CPU and reported the anomaly on 24 October 1994.

Localization

Norwegian engineer Terje Mathisen reproduced the discrepancy with a short assembly test case posted to the comp.sys.intel newsgroup. German researcher Andreas Kaiser identified more than twenty specific numbers whose reciprocals lost precision, reducing to single‑precision (≈7 decimal digits) accuracy.

Tim Coe, an FPU designer at Vitesse Semiconductor, analyzed the reported numbers and inferred that the Pentium division unit used a radix‑4 SRT algorithm that produces two quotient bits per clock cycle. He built a mathematical model of the divider and highlighted a worst‑case example: 4195835 / 3145727 The example demonstrates the maximum error observed for the faulty lookup entries. Cleve Moler later noted that all problematic divisors share a simple pattern: they are slightly less than three times a power of two (i.e., divisor ≈ 3·2^n – ε).

Fix

A collaborative effort (Moler, Coe, Mathisen, Intel engineers, and others) produced a software‑level mitigation that can be applied before each division:

Extract the high four bits of the divisor’s fraction field (the 23‑bit mantissa of a 32‑bit IEEE‑754 single‑precision float).

If the pattern matches any of 0001, 0100, 0111, 1010, or 1101, multiply both dividend and divisor by the scaling factor 15/16 (i.e., ×0.9375).

This scaling moves the divisor out of the “dangerous” region, guaranteeing that the subsequent division yields the correct quotient.

The method was later refined to examine the high eight bits of the fraction. The scaling is applied only when the eight‑bit pattern equals one of:

00011111

01001111

01111111

10101111

11011111

Checking eight bits reduces the number of unnecessary scalings and therefore the performance overhead.

The fix was distributed publicly via the newsgroup and can be incorporated into any software that performs floating‑point division on affected Pentium CPUs.