Microsecond‑Level Sleep: Why thread::sleep Misses Its Target and How to Fix It

Why thread::sleep Is Not Precise

The Rust standard library documents thread::sleep as sleeping for at least the requested duration, never less. On Linux this maps to a kernel call clock_nanosleep that rounds the timeout up to the nearest timer granularity and then adds the per‑thread timer_slack (default 50 µs) before the scheduler can wake the thread.

In practice, std::thread::sleep(Duration::from_micros(2)) sleeps ~60 µs, about 30× the requested time, and changing the request to 1 µs or 5 µs yields almost the same result.

Because the API promises only a lower bound, any code that expects microsecond‑level accuracy is built on a wrong premise; the actual latency depends on OS load, CPU idle state, and scheduler decisions.

Empirical Measurements

Benchmarks on an AWS EC2 instance (AMD EPYC, Linux 6.1, Rust release) with 100 000 samples per point show a stable floor around 59 µs regardless of the requested sleep time. Only when the request reaches 10 µs does the measured p50 rise proportionally (≈ 9 µs extra). The data (p50/p99/max) reveal three counter‑intuitive facts:

The overhead is independent of the requested duration for sub‑10 µs values.

Only the 10 µs bucket shows a measurable increase matching the extra request.

Occasional minima of 11–20 µs occur when the thread happens to catch a completely idle CPU.

The Real Culprit: timer_slack

Since Linux 2.6.28 each thread has timer_slack_ns defaulting to 50 µs. When a blocking call with a timeout (e.g., nanosleep) is used, the kernel may delay the wake‑up by up to one slack interval to coalesce nearby timers and save power. This explains the ~59 µs floor (50 µs slack + ~9 µs wake‑up/scheduling overhead).

Reducing timer_slack to 1 ns with the userspace call

unsafe { libc::prctl(libc::PR_SET_TIMERSLACK, 1u64, 0, 0, 0); }

drops the floor to ~10 µs. Pinning the thread to a core (CPU affinity) does not affect the floor; it only reduces tail jitter.

Busy‑Spin for Sub‑µs Accuracy

When sub‑µs latency is required (e.g., high‑frequency trading), busy‑spin is the only viable path because it stays entirely in user space and avoids the kernel round‑trip. A minimal spin loop looks like:

use std::{hint::spin_loop, time::{Duration, Instant}};
fn spin_sleep(dur: Duration) {
    let deadline = Instant::now() + dur;
    while Instant::now() < deadline { spin_loop(); }
}

On x86 the spin_loop() compiles to a single PAUSE instruction, which yields the pipeline without leaving the CPU. The time check uses the vDSO clock_gettime (≈ 20 ns when the TSC is the clocksource), making the loop cheap enough to stay within a few microseconds.

Rigtorp’s Low‑Latency Guide repeatedly stresses the rule “never enter the kernel for precise waiting”. The cost of spinning is 100 % CPU, higher power draw, and the need for isolation (e.g., isolcpus, nohz_full) and disabling RT throttling ( kernel.sched_rt_runtime_us=-1). Shutdown handling also requires special care because a pure spin thread does not receive a SIGKILL reliably.

A hybrid approach—sleep the coarse part then spin the remainder—can be implemented as:

fn hybrid_wait(dur: Duration) {
    if let Some(coarse) = dur.checked_sub(Duration::from_micros(60)) {
        std::thread::sleep(coarse);
    }
    while Instant::now() < Instant::now() + dur { spin_loop(); }
}

When the target is much larger than the ~10 µs floor, the hybrid method (or the community spin_sleep crate) gives good accuracy with lower CPU usage. For targets ≤ 10 µs the hybrid degrades to pure spin.

Full‑System Tuning Checklist

isolcpus=8-31

– remove cores from the OS scheduler’s load‑balancer. nohz_full=8-31 – disable periodic timer ticks on isolated cores. rcu_nocbs=8-31 – keep RCU callbacks off those cores. processor.max_cstate=1 – lock C‑states to C1, eliminating ~800 µs C2 exit latency. transparent_hugepage=never – avoid THP‑induced jitter.

Sysctl: kernel.sched_rt_runtime_us=-1 (disable RT throttling), vm.swappiness=1, kernel.numa_balancing=0, kernel.nmi_watchdog=0, NIC LRO/GRO=off, NIC IRQ affinity = pin to isolated core, optionally enable busy_poll.

Process‑level: set real‑time policy SCHED_FIFO (prio 95) which zeroes timer_slack, lock memory with mlockall(MCL_CURRENT|MCL_FUTURE), and pin the thread with sched_setaffinity. Also call prctl(PR_SET_TIMERSLACK, 1) as a safety net.

Note that isolcpus+nohz_full creates a core ideal for spinning but not for sleeping, because the kernel‑side timer path is still traversed.

Production Validation

On a 32‑vCPU machine already tuned for low latency, a read‑only benchmark showed:

Non‑pinned core: p50 ≈ 9.8 µs, p99 ≈ 16.7 µs, max ≈ 7.6 ms (large tail).

Isolated core (CPU 31) with the above sysctl tweaks: p50 ≈ 11 µs, p99 ≈ 11.7 µs, max ≈ 25 µs (≈ 300× tail reduction).

Pure spin on the same isolated core: p50 ≈ 2 µs, p99 ≈ 2 µs, max ≈ 16 µs.

The tail improvement is the most valuable outcome for quant‑trading systems where predictability outweighs raw median latency. The slight increase in p50 on the isolated core is explained by the extra ~1 µs C1 exit latency and the higher accounting cost of nohz_full when the core frequently enters and leaves idle.

Decision Guide

Background polling, rate‑limiting, tens‑of‑ms timers – use thread::sleep. Accuracy is sufficient and CPU usage is low.

ms‑level delays tolerating ± tens µs jitter – use sleep + PR_SET_TIMERSLACK. Simple and low CPU.

Hundreds µs – tens ms precise waits – use the spin_sleep crate (hybrid). Low CPU, high accuracy.

µs / sub‑µs waits (HFT hot path) – use pure busy‑spin . Sleep cannot reach this floor; spin is the only solution.

Event waiting with ultra‑low latency notification – combine spin + try_recv. End‑to‑end ~50‑150 ns, far better than sleep.

Three Takeaways

sleep(2µs)

can never reliably sleep 2 µs; the OS guarantees only a lower bound (≈ 60 µs default, ≈ 10 µs after timer_slack reduction).

The ~60 µs floor is caused by timer_slack, not by missing CPU affinity. Setting PR_SET_TIMERSLACK=1 or using SCHED_FIFO brings the floor down to ~10 µs; pinning only reduces tail jitter.

Sub‑µs precision requires pure busy‑spin with full core isolation, RT throttle disabled, and proper shutdown handling; the ~10 µs sleep floor is a hard physical limit.