Intranode Latency in High‑Frequency Trading: From NIC to Strategy

Why Intranode Latency Matters

Geographic distance, edge nodes and CDNs are important for the Internet, but for high‑frequency trading (HFT) teams that already have colocated servers and dedicated lines, the decisive factor is how fast and stable a packet moves once it is inside the machine.

Critical Path Inside a Linux Host

NIC : packet enters the receive (RX) queue.

IRQ : the NIC raises a hard interrupt or works with a polling mechanism to notify the CPU.

Kernel (soft‑irq) : the driver hands the packet to the network stack, entering the NAPI/soft‑irq flow.

Protocol Stack : L2/L3/L4 parsing, checksum, reassembly, etc.

Socket : data is placed in the socket receive buffer and wakes the waiting thread or triggers epoll.

User‑space : the application calls recv() / read(), parses market data, runs strategy or matching logic, then uses send() to place an order.

Each layer can introduce queuing, scheduling, cache misses and jitter; HFT cares not only about lower average latency but also about shorter, more predictable tail latency.

Interrupts, Soft‑Interrupts and NAPI

Hard Interrupt (IRQ)

When a packet arrives, the NIC can fire a hard interrupt to tell the CPU “a packet is ready”. If every packet caused an IRQ, the CPU would be overwhelmed (interrupt storm) and throughput would collapse. Linux therefore uses the NAPI model: it polls the NIC under high load and falls back to interrupts under low load.

Hard interrupts bring two risks for low‑latency trading:

Jitter : the interrupt pre‑empts the current execution flow at an unpredictable moment.

Interference : if the interrupt hits the core strategy thread, the thread is needlessly interrupted.

Soft‑Interrupt (softirq)

Most network‑stack processing happens in the soft‑irq context (e.g., NET_RX_SOFTIRQ). It is not a normal process context; its scheduling depends on the kernel’s policy, CPU load and affinity. A soft‑irq that runs too long can push other tasks aside, creating tail‑latency spikes. In practice, occasional P99/P999 latency spikes often trace back to a soft‑irq that stayed too long on a CPU or migrated to an unsuitable core.

Interrupt‑to‑CPU Affinity

Two common CPU‑planning approaches are:

Isolate : dedicate cores for NIC IRQ/softirq and run strategy threads on a separate set of cores. Pros : strategy cores stay clean. Cons : cross‑core communication between protocol handling and user‑space adds cache‑coherency traffic, especially on NUMA systems.

Colocate : bind the IRQ/softirq that serves a specific RX queue to the same core that runs the application thread handling that flow. Pros : reduces cross‑core migration, potentially lowering tail latency. Cons : requires precise mapping of queues to cores.

HFT systems aim for determinism: the same market‑data flow and order‑submission path should repeatedly execute on the same core, forming a closed loop from packet receipt to order transmission.

Multi‑Queue (RSS) and Application‑Level Partitioning

Modern NICs provide multiple RX queues and use Receive Side Scaling (RSS) to hash the five‑tuple and distribute flows across queues. This determines which queue a market‑data feed lands in, whether a TCP connection consistently hits the same queue, and which CPU processes the associated soft‑irq.

A common design is application‑level partitioning: split traffic by connection, instrument or session, then bind “queue → CPU → application thread” one‑to‑one. Each thread processes only its assigned flows, avoiding lock contention on shared queues. The same idea applies to matching or risk modules, partitioning orders by account or instrument.

TCP Nagle and Its Hidden Queueing

Nagle Algorithm

When unacknowledged data is in flight, small writes may be buffered until an ACK arrives or enough data accumulates.

While Nagle improves throughput for many small packets, it introduces conditional queuing latency—an “tail‑latency amplifier”—which is disastrous for HFT where messages are tiny and must be sent immediately.

Disabling Nagle

Trading systems that use TCP almost always set TCP_NODELAY: setsockopt(fd, IPPROTO_TCP, TCP_NODELAY, ...) This forces send() to transmit packets immediately. The trade‑off is a higher packet rate (more network and NIC load) and the fact that TCP’s own reliability mechanisms can still produce long tails under loss or congestion.

Using UDP

UDP is common for market‑data broadcast and intra‑process low‑latency messaging in reliable network environments. Its advantages are a simpler stack, no connection state and no retransmission, yielding lower latency and shorter paths. The downside is that lost packets are lost; applications must tolerate occasional loss or implement their own sequencing, replay or error‑correction mechanisms and monitor loss, reordering and gap‑recovery times.

Kernel‑Bypass, TOE and XDP

When IRQ/softirq tuning, Nagle disabling and lock reduction are insufficient, more aggressive techniques are considered:

User‑space protocol stack / kernel bypass (e.g., DPDK, Netmap): packets are delivered directly to user‑space memory, eliminating system calls and kernel overhead.

Hardware offload (TOE / SmartNIC / FPGA) : TCP processing or even parts of business logic are offloaded to the NIC or dedicated hardware, pushing microsecond‑level jitter lower.

XDP/eBPF : early packet processing before the full network stack (filtering, forwarding, timestamping, mirroring) can dramatically reduce the number of packets that enter the “heavy” stack, thus reducing jitter.

All these methods share the goal of reducing uncontrolled overhead in the generic kernel path, at the cost of higher engineering complexity, maintainability challenges, and stricter compatibility and operations requirements.

Practical Optimization Roadmap

Identify the critical path : measure market‑data receipt, strategy decision and order transmission separately rather than only end‑to‑end.

Thread‑CPU affinity : bind strategy threads to dedicated cores, separate them from IRQ/softirq and background tasks.

IRQ/softirq planning : decide which cores handle interrupts and ensure queue‑thread alignment.

TCP latency traps : enable TCP_NODELAY and evaluate socket buffers, queue lengths and other socket options.

Reduce sharing and locking : partition by connection, instrument or account to keep each thread’s workload isolated.

Consider bypass/offload when needed : adopt DPDK/user‑space stacks, SmartNICs or XDP if the kernel path remains a bottleneck.

The overall conclusion is that shaving latency in HFT is less about buying faster CPUs and more about converting a general‑purpose system into a purpose‑built, deterministic pipeline.