How to Build a Low‑Latency C++ Trading System for a Million‑Dollar Quant Interview

Low‑latency definition in HFT

In high‑frequency trading (HFT) latency is measured in microseconds (µs) or nanoseconds (ns) rather than milliseconds. Interviewers focus on three dimensions:

Time‑scale difference : Internet RTT is 20‑100 ms, but kernel‑bypass can compress network‑to‑application latency to 1‑5 µs. A typical mutex wake‑up costs ~15 µs, while a well‑tuned spin‑lock can be ~300 ns.

Determinism vs. tail latency : Average latency is meaningless; a single 500 µs outlier can cause loss. Candidates must discuss controlling the 99th‑percentile (P99) or P99.99 latency by eliminating OS scheduling, memory‑allocation jitter, and cache‑miss jitter.

Throughput vs. latency : Unlike generic web servers that batch to increase throughput, a low‑latency trading engine deliberately sacrifices throughput to keep each trade’s latency minimal.

General‑system thinking : keep the CPU fully utilized.

HFT‑system thinking : pin a core and busy‑spin so the thread can react instantly to market data.

Memory model & hardware affinity

Cache hierarchy and false sharing

Typical cache‑miss costs are:

L1 cache : ~3‑4 cycles (≈1 ns)

L2 cache : ~10‑12 cycles (≈3‑4 ns)

L3 cache : ~30‑70 cycles (≈10‑20 ns)

Main RAM : ~300+ cycles (≈60‑100 ns)

Keeping hot data in L1/L2 is critical. False sharing occurs when independent variables share the same 64‑byte cache line, causing the MESI protocol to invalidate the line on each write and producing severe jitter.

Interview strategy : When asked why not use std::list or std::map , explain that pointer chasing destroys spatial locality and forces the prefetcher to fail. A std::vector preserves contiguity.

Alignment and padding

struct SharedData {
    std::atomic<int> head; // Thread A writes here
    std::atomic<int> tail; // Thread B writes here
    // head and tail may reside on the same 64‑byte cache line
};

struct AlignedData {
    alignas(64) std::atomic<int> head;
    alignas(64) std::atomic<int> tail;
};

Since C++17 the constant std::hardware_destructive_interference_size provides the optimal alignment size (typically 64 bytes).

NUMA architecture and CPU pinning

In multi‑socket servers each socket has local memory. Accessing remote memory adds latency and consumes inter‑socket bandwidth.

Latency increase : remote memory access is slower.

Bandwidth contention : the inter‑connect can become a bottleneck.

Trading threads must run on the same NUMA node as their memory and NIC. Linux implementations typically use isolcpus to remove a set of cores from the OS scheduler and then pthread_setaffinity_np to bind the thread.

// Bind current thread to core 2
cpu_set_t cpuset;
CPU_ZERO(&cpuset);
CPU_SET(2, &cpuset);
pthread_t cur = pthread_self();
int result = pthread_setaffinity_np(cur, sizeof(cpuset), &cpuset);
if (result != 0) {
    // handle error: binding failed, latency determinism cannot be guaranteed
}

Lock‑free programming

Using std::mutex on the hot path introduces contention‑driven context switches and cache pollution. A lock‑free design avoids thread suspension.

Linux kernel benchmark: lock contention can push the 99th‑percentile latency to 1.4 ms, while a lock‑free implementation stays around 0.07 ms.

Lock‑free vs. wait‑free

Lock‑free : at least one thread makes progress; starvation possible.

Wait‑free : every thread completes in a bounded number of steps; the “golden” goal for HFT.

Memory‑order defaults

The default memory_order_seq_cst inserts full memory fences (MFENCE or LOCK prefixes) that cost dozens of cycles on x86 and even more on weak‑memory architectures. Prefer memory_order_release on stores and memory_order_acquire on loads, falling back to seq_cst only when a global total order is required.

// Producer
payload = 42;
flag.store(true, std::memory_order_release);

// Consumer
while (!flag.load(std::memory_order_acquire));
assert(payload == 42);

memory_order_relaxed

is safe only for operations that have no ordering constraints, e.g., reference‑count increments.

SPSC ring‑buffer (lock‑free queue)

Design a fixed‑size array with separate head (producer) and tail (consumer) indices. No CAS is needed because each side writes only its own index.

struct RingBuffer {
    alignas(64) std::atomic<size_t> head;
    char padding1[64 - sizeof(std::atomic<size_t>)];
    alignas(64) std::atomic<size_t> tail;
    char padding2[64 - sizeof(std::atomic<size_t>)];
    Element buffer[CAPACITY];
};

Producer stores data, then head.store(new_head, std::memory_order_release). Consumer loads head.load(std::memory_order_acquire), reads the element, then updates tail. Using relaxed for the tail store would allow the consumer to see a stale value.

System & network optimisation

Kernel bypass & zero‑copy

Standard TCP/IP processing involves interrupt, context switch, and memory copy, adding tens of microseconds. Bypass solutions such as Solarflare OpenOnload (user‑space library) or DPDK (poll‑mode drivers) let the NIC DMA packets directly into user‑space buffers, reducing round‑trip time to 1‑5 µs.

Busy‑spinning vs. interrupt

In a low‑latency engine the thread busy‑spins on the NIC receive queue, consuming 100 % of a core but eliminating wake‑up latency.

CPU affinity for the polling thread

Pin the polling core, disable hyper‑threading, and keep the NIC on the same NUMA node to avoid cross‑socket traffic and cache‑pollution caused by OS scheduling.

Interview knowledge map

Candidates should be able to discuss cache hierarchy, false sharing, NUMA, kernel bypass, lock‑free data structures, memory‑order semantics, and modern C++ features such as constexpr, CRTP, and [[likely]] / [[unlikely]] that together enable microsecond‑ or nanosecond‑scale performance.