7 Linux Bonding Modes Explained: From Configuration to Kernel Tuning

During years of operations work, many outages have been caused by improper NIC configuration; a single‑NIC failure once crippled a payment system during a major sale despite the server having two NICs. This article provides a comprehensive, hands‑on guide to Linux NIC bonding.

What is NIC bonding and why use it

NIC bonding combines multiple physical NICs into one logical interface, offering high availability (automatic failover) and higher throughput through load‑balancing. It is widely used on database servers, load balancers, and storage nodes, especially for 10 GbE links where a single NIC cannot saturate the bandwidth.

Seven bonding modes and their characteristics

The Linux kernel bonding driver supports seven modes, each with different switch requirements and performance traits. Selecting the wrong mode can render bonding ineffective or even degrade performance.

Mode 0: balance‑rr (round‑robin)

Packets are sent alternately on each NIC (eth0, eth1, eth0, eth1…). This yields simple load‑balancing and bandwidth aggregation, but TCP packets of the same connection may arrive out of order, increasing CPU overhead for reassembly. It also requires switch port‑aggregation; without it, a broadcast storm occurs.

Use case: High‑performance environments where the switch can be configured for port aggregation, such as internal test clusters.

Mode 1: active‑backup

Only the primary NIC is active; the backup NIC monitors the link and takes over within milliseconds on failure. No special switch configuration is needed, but bandwidth is limited to a single NIC, wasting about 50 % of the hardware.

Use case: Scenarios prioritising stability over bandwidth, e.g., management or monitoring networks.

Mode 4: 802.3ad (LACP dynamic link aggregation)

This is the most common production mode. Using LACP, the NICs and switch negotiate an aggregated link that provides both load‑balancing and transparent failover. It avoids out‑of‑order packets because a TCP flow is bound to a single physical link.

Use case: Critical production workloads such as databases, caches, and load balancers.

Other modes (2, 3, 5, 6)

Mode 2 – balance‑xor: MAC‑address XOR hash; requires static switch aggregation.

Mode 3 – broadcast: Sends every packet on all NICs; wastes bandwidth and is rarely used.

Mode 5 – balance‑tlb: Adaptive transmit load‑balancing; no switch configuration needed.

Mode 6 – balance‑alb: Adds receive load‑balancing via ARP negotiation; most flexible but higher overhead.

CentOS/RHEL configuration with NetworkManager

CentOS 7/8 and RHEL use NetworkManager. The nmcli commands below create a bond interface, add physical NICs, assign an IP address, and activate the configuration.

nmcli connection add type bond ifname bond0 mode 802.3ad
nmcli connection add type ethernet ifname eth0 master bond0
nmcli connection add type ethernet ifname eth1 master bond0
nmcli connection modify bond0 ipv4.addresses 192.168.1.10/24
nmcli connection modify bond0 ipv4.gateway 192.168.1.1
nmcli connection modify bond0 ipv4.dns 8.8.8.8
nmcli connection modify bond0 ipv4.method manual
nmcli connection up bond0

After activation, run cat /proc/net/bonding/bond0 to verify that both slaves are listed and their MII status is up.

Ubuntu/Debian configuration with Netplan

Ubuntu 18.04+ uses Netplan (YAML). Edit /etc/netplan/00-installer-config.yaml as follows (pay attention to indentation):

network:
  version: 2
  ethernets:
    eth0: {}
    eth1: {}
  bonds:
    bond0:
      interfaces: [eth0, eth1]
      parameters:
        mode: 802.3ad
        mii-monitor-interval: 100
        transmit-hash-policy: layer3+4
      addresses: [192.168.1.10/24]
      gateway4: 192.168.1.1
      nameservers:
        addresses: [8.8.8.8, 114.114.114.114]

The mii-monitor-interval of 100 ms detects link failures quickly. The transmit-hash-policy of layer3+4 hashes on IP and port, keeping a TCP flow on the same physical link and avoiding packet reordering.

Apply the configuration with netplan apply. If the configuration is wrong, Netplan rolls back after 120 seconds.

Queue and Ring Buffer tuning

After bonding, adjust the NIC’s queue count and ring buffer size to unlock full performance.

Check current queues: ethtool -l eth0 If the "Combined" value is far below the hardware maximum, increase it, e.g.: ethtool -L eth0 combined 32 This enables 32 CPU cores to handle interrupts simultaneously.

Inspect ring buffer settings: ethtool -g eth0 Resize both receive and transmit buffers: ethtool -G eth0 rx 4096 tx 4096 In a video‑forwarding server, raising the ring buffer from 256 to 4096 reduced packet loss from 3 % to below 0.1 %.

Interrupt affinity optimization

By default, all NIC interrupts may land on CPU 0, creating an "interrupt skew" bottleneck. Verify with: watch -n1 'cat /proc/interrupts | grep eth' If all eth0‑TxRx‑* entries show activity only on the first CPU column, apply one of the following solutions.

Option 1 – irqbalance service: Enable and start the service to automatically distribute interrupts.

systemctl enable irqbalance
systemctl start irqbalance

Option 2 – manual binding: Write a hexadecimal CPU mask to each IRQ file, e.g.:

echo '02' > /proc/irq/47/smp_affinity   # CPU1
echo '04' > /proc/irq/48/smp_affinity   # CPU2
echo '08' > /proc/irq/49/smp_affinity   # CPU3

The masks 02, 04, 08 correspond to CPUs 1, 2, 3 respectively.

Kernel network parameter tuning

Linux defaults are conservative for high‑concurrency workloads. The following sysctl settings are recommended:

net.core.somaxconn = 65535
net.core.netdev_max_backlog = 5000
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
net.ipv4.tcp_congestion_control = bbr
net.core.default_qdisc = fq

somaxconn

raises the TCP listen queue from the default 128 to the maximum 65535, preventing connection refusals under load. The tcp_rmem and tcp_wmem triples set minimum, default, and maximum buffer sizes; increasing the maximum to 16 MiB improves throughput for large transfers. Switching the congestion control algorithm to bbr and the queue discipline to fq yields higher throughput, especially on lossy links.

Append the settings to /etc/sysctl.conf and apply with sysctl -p.

Summary

NIC bonding is not a set‑and‑forget task. Choosing the proper mode, tuning queues and ring buffers, fixing interrupt affinity, and adjusting kernel network parameters each have a measurable impact on final performance.

Production‑grade 802.3ad mode offers both high availability and load‑balancing.

Adjust queue count and Ring Buffer according to hardware capacity and workload.

Optimise interrupt affinity to avoid a CPU‑0 bottleneck; irqbalance is the simplest method.

Combine BBR congestion control with a large memory window to multiply internal‑network throughput.

Next issue: advanced Linux firewall techniques with iptables and nftables.