Balancing Hardware Interrupts Across CPUs with SMP IRQ Affinity on Linux

Frequent hardware interrupts consume a lot of CPU resources; on multi‑core CPUs, distributing these interrupts across different cores can balance load and boost performance.

Modern servers often have multiple CPUs, NICs, and disks, so assigning a network card’s interrupts to one core and disk I/O interrupts to another can significantly reduce the burden on any single CPU and improve overall efficiency.

What Is an Interrupt

An interrupt is a signal from a hardware device (e.g., disk, NIC) that tells the CPU something has happened, causing the CPU to pause its current work and handle the event.

Each hardware device needs a way to communicate with the CPU so the CPU can react promptly; this hardware‑initiated communication is called a hardware interrupt.

It is similar to being interrupted by a chat message while working.

Interrupts are more efficient than polling, where the CPU periodically checks device status.

Each device is assigned a unique IRQ number, allowing the system to distinguish the source of an interrupt.

Interrupt signals are generated by hardware, sent to an interrupt controller (e.g., 8259A or APIC), which then notifies the CPU.

Two common interrupt controllers are the programmable interrupt controller (8259A) for single‑CPU systems and the advanced programmable interrupt controller (APIC) for SMP (multi‑CPU) systems.

Only Linux kernels newer than 2.4 support binding specific IRQs to particular CPUs; this feature is called SMP IRQ Affinity.

For more details see the kernel documentation: linux-2.6.31.8/Documentation/IRQ-affinity.txt

How to Use SMP IRQ Affinity

First, view the current IRQ distribution:

# cat /proc/interrupts
            CPU0       CPU1
  0:  918926335          0   IO-APIC-edge  timer
  1:          2          0   IO-APIC-edge  i8042
  8:          0          0   IO-APIC-edge  rtc
  9:          0          0   IO-APIC-level acpi
 12:          4          0   IO-APIC-edge  i8042
 14:   8248017          0   IO-APIC-edge  ide0
 50:        194          0   IO-APIC-level ohci_hcd:usb2
 58:       31673          0   IO-APIC-level sata_nv
 90:    1070374          0   PCI-MSI       eth0
233:         10          0   IO-APIC-level ehci_hcd:usb1
NMI:       5077       2032
LOC:  918809969  918809894
ERR:          0
MIS:          0

To move part of the load to CPU1, check the current affinity mask for IRQ 90 (the NIC):

# cat /proc/irq/90/smp_affinity
7fffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff

Stop the automatic IRQ balancing service before making manual changes: # /etc/init.d/irqbalance stop Bind IRQ 90 to CPU1 (the second CPU) by writing the appropriate mask ("2"): # echo "2" > /proc/irq/90/smp_affinity After a while, monitor /proc/interrupts to see the effect. The NIC’s interrupt count on CPU1 should increase while CPU0 remains unchanged:

# cat /proc/interrupts
            CPU0       CPU1
  0:  922506515          0   IO-APIC-edge  timer
  1:          2          0   IO-APIC-edge  i8042
  8:          0          0   IO-APIC-edge  rtc
  9:          0          0   IO-APIC-level acpi
 12:          4          0   IO-APIC-edge  i8042
 14:   8280147          0   IO-APIC-edge  ide0
 50:        194          0   IO-APIC-level ohci_hcd:usb2
 58:       31907          0   IO-APIC-level sata_nv
 90:    1073399        145   PCI-MSI       eth0
233:         10          0   IO-APIC-level ehci_hcd:usb1
NMI:       5093       2043
LOC:  922389696  922389621
ERR:          0
MIS:          0

Why It Matters

For heavily loaded network services, file servers, or high‑traffic web servers, distributing NIC IRQs across CPUs reduces contention and improves overall interrupt handling capacity.

On database servers, binding disk controller IRQs to one CPU and NIC IRQs to another can lower response times and enhance performance.

Balancing IRQs according to the specific workload helps increase overall system throughput and efficiency.

Performance tuning is an ongoing process that requires testing, monitoring, and continuous adjustment, not just a one‑time configuration change.