How Linux Processes Network Packets: From NIC Interrupts to the Protocol Stack

Linux Network Subsystem Overview

The Linux networking stack is built on a layered design that isolates protocol families (INET, INET6, UNIX, NETLINK), network devices, and the BSD socket API, providing a uniform interface for user‑space applications.

TCP/IP Layer Model

In the TCP/IP model the stack is divided into Physical, Link, Network, Transport, and Application layers. Linux implements the Link, Network, and Transport layers, exposing a socket interface to user space.

Linux Network Stack Diagram

Interrupt Handling Basics

Hardware interrupts notify the CPU of events such as packet arrival. To keep interrupt latency low, the handling is split into a fast top half (hardware interrupt) and a deferred bottom half (soft‑irq, tasklet, or workqueue).

Soft‑IRQ Registration

enum {</code><code>    HI_SOFTIRQ=0,</code><code>    TIMER_SOFTIRQ,</code><code>    NET_TX_SOFTIRQ,</code><code>    NET_RX_SOFTIRQ,</code><code>    BLOCK_SOFTIRQ,</code><code>    IRQ_POLL_SOFTIRQ,</code><code>    TASKLET_SOFTIRQ,</code><code>    SCHED_SOFTIRQ,</code><code>    HRTIMER_SOFTIRQ,</code><code>    RCU_SOFTIRQ,</code><code>    NR_SOFTIRQS</code><code>};

Network drivers register NET_RX_SOFTIRQ and NET_TX_SOFTIRQ to handle received and transmitted packets.

Driver Example: e1000

static int __init e1000_init_module(void)</code><code>{</code><code>    int ret;</code><code>    pr_info("%s - version %s
", e1000_driver_string, e1000_driver_version);</code><code>    ret = pci_register_driver(&e1000_driver);</code><code>    ...</code><code>    return ret;</code><code>}

The driver’s probe routine registers a net_device and sets netdev_ops:

static const struct net_device_ops e1000_netdev_ops = {</code><code>    .ndo_open            = e1000_open,</code><code>    .ndo_stop            = e1000_close,</code><code>    .ndo_start_xmit      = e1000_xmit_frame,</code><code>    .ndo_set_rx_mode     = e1000_set_rx_mode,</code><code>    .ndo_set_mac_address = e1000_set_mac,</code><code>    .ndo_tx_timeout      = e1000_tx_timeout,</code><code>    ...</code><code>};

The e1000_open function registers the hardware interrupt:

int e1000_open(struct net_device *netdev)</code><code>{</code><code>    struct e1000_adapter *adapter = netdev_priv(netdev);</code><code>    ...</code><code>    err = e1000_request_irq(adapter);</code><code>    ...</code><code>}

static int e1000_request_irq(struct e1000_adapter *adapter)</code><code>{</code><code>    irq_handler_t handler = e1000_intr;</code><code>    int irq_flags = IRQF_SHARED;</code><code>    err = request_irq(adapter->pdev->irq, handler, irq_flags, netdev->name, ...);</code><code>    ...</code><code>}

Soft‑IRQ Bottom Half

When a packet arrives, the NIC raises a hardware interrupt, which quickly disables further NIC interrupts and schedules the soft‑irq NET_RX_SOFTIRQ. The kernel thread ksoftirqd processes this soft‑irq.

ksoftirqd Thread Creation

static struct smp_hotplug_thread softirq_threads = {</code><code>    .store          = &ksoftirqd,</code><code>    .thread_should_run = ksoftirqd_should_run,</code><code>    .thread_fn      = run_ksoftirqd,</code><code>    .thread_comm    = "ksoftirqd/%u",</code><code>};

The thread repeatedly checks local_softirq_pending() and calls __do_softirq() when work is pending.

static void run_ksoftirqd(unsigned int cpu)</code><code>{</code><code>    local_irq_disable();</code><code>    if (local_softirq_pending()) {</code><code>        __do_softirq();</code><code>        rcu_note_context_switch(cpu);</code><code>        local_irq_enable();</code><code>        cond_resched();</code><code>        return;</code><code>    }</code><code>    local_irq_enable();</code><code>}

Network Startup Preparation

Create ksoftirqd threads (one per CPU).

Register protocol handlers (ARP, ICMP, IP, UDP, TCP) via inet_init which populates inet_protos and ptype_base.

Initialize NIC drivers, allocate DMA buffers, and set up NAPI poll functions.

Allocate RX/TX queues and register interrupt handlers.

Soft‑IRQ Processing Path

For NAPI‑enabled NICs (e.g., e1000):

net_rx_action() → e1000_clean() → e1000_clean_rx_irq() → e1000_receive_skb() → netif_receive_skb()

For non‑NAPI NICs (e.g., dm9000):

net_rx_action() → process_backlog() → netif_receive_skb()

The function net_rx_action limits its execution with a time budget and packet budget to avoid monopolizing the CPU.

static void net_rx_action(struct softirq_action *h)</code><code>{</code><code>    struct softnet_data *sd = &__get_cpu_var(softnet_data);</code><code>    unsigned long time_limit = jiffies + 2;</code><code>    int budget = netdev_budget;</code><code>    ...</code><code>}

Packet Reception Flow

The NIC writes incoming frames to its FIFO and DMA copies them into sk_buff buffers allocated in a ring.

A hardware interrupt fires; the top half disables further NIC interrupts and raises NET_RX_SOFTIRQ. ksoftirqd runs net_rx_action, which invokes the driver’s NAPI poll function (e.g., igb_poll).

The poll function processes descriptors, builds sk_buff s, and calls napi_gro_receive → netif_receive_skb to hand the packet to the IP/TCP/UDP layers.

Packet Transmission Flow

The driver creates a TX descriptor ring and writes its bus address to the NIC.

User‑space calls dev_queue_xmit(), which builds an sk_buff and passes it to the driver via ndo_start_xmit.

The driver fills TX descriptors, updates the tail pointer, and the NIC’s DMA engine copies the packet to the TX FIFO.

The NIC transmits the frame and raises a completion interrupt; the driver frees the descriptors.

Key Takeaways

Linux separates fast interrupt handling from the heavier packet processing using soft‑irqs and the ksoftirqd kernel thread.

Proper configuration of IRQ affinity and NAPI poll budgets is essential for high‑throughput networking.

Understanding the driver registration flow ( pci_register_driver, net_device_ops, NAPI) helps when debugging packet loss or latency issues.