Nanosecond‑Level Parsing: Zero‑Copy Rust Parsing in HFT Systems

1. Market‑Data Parsing Hot Path

In high‑frequency trading each nanosecond matters. A typical aggTrade push from an exchange looks like:

{
  "stream":"solusdt@aggTrade",
  "data":{
    "e":"aggTrade",
    "E":1745838069228,
    "a":893418959,
    "s":"SOLUSDT",
    "p":"152.3300",
    "q":"0.06",
    "f":2271916894,
    "l":2271916894,
    "T":1745838069116,
    "m":false
  }
}

Parsing this tiny JSON dozens of thousands of times per second with the textbook approach hits three pitfalls:

Repeated heap allocation because serde_json::from_str creates a String for every field.

Redundant boundary checks in hand‑written state machines.

Multiple copies: socket buffer → String → struct.

2. Zero‑Copy Core Idea

The essential principle is to never copy the source data; the parser returns a “borrowed view” (pointer + length) into the original buffer.

2.1 Lexer Holds Only Offsets

pub struct Lexer<'source, Token: Logos<'source>> {
    source: &'source Token::Source,
    token: core::mem::ManuallyDrop<Option<Result<Token, Token::Error>>>,
    token_start: usize,
    token_end: usize,
    pub extras: Token::Extras,
}

Key points: source borrows the input buffer for the lexer’s entire lifetime; no byte is ever copied.

The parsing state reduces to two usize offsets; scanning simply moves these cursors. ManuallyDrop avoids an extra write‑and‑drop of the temporary token field.

2.2 Slice – Borrowing a Token

pub fn slice(&self) -> <Token::Source as Source>::Slice<'source> {
    unsafe { self.source.slice_unchecked(self.span()) }
}

For a str source the return type is &'source str, i.e. a borrowed slice that points directly into the original socket buffer.

2.3 Batch Reads and Eliminating Bounds Checks

fn read<'a, Chunk>(&self, offset: usize) -> Option<Chunk>
where
    Chunk: self::Chunk<'a>,
{ /* single boundary check for the whole chunk */ }

Two HFT‑friendly optimisations are hidden here:

Batch reading a fixed‑size Chunk performs only one if offset + N - 1 < len check for the whole chunk. from_ptr casts a *const u8 directly to &[u8; N], allowing the compiler to emit a single wide‑load instruction.

When compiled with derive(Logos) all token rules are merged into a deterministic finite automaton (DFA); the generated code is usually tighter and has fewer branches than a hand‑written match state machine.

2.4 ManuallyDrop Saves One Take

The iterator’s next method updates token_start, runs the generated DFA, and then uses ManuallyDrop::take (or Option::take when forbid_unsafe is enabled) to avoid an extra write‑and‑drop of the temporary token field.

3. Practical Example: Zero‑Copy aggTrade Parser

Step 1 – Define Tokens for Fixed Keys

use logos::Logos;

#[derive(Logos, Debug, PartialEq)]
#[logos(skip r"[ {},:\t
\f]+")]
enum AggTrade {
    #[token("\"e\"")] EventTag,
    #[token("\"E\"")] EventTimeTag,
    #[token("\"a\"")] AggIdTag,
    #[token("\"s\"")] SymbolTag,
    #[token("\"p\"")] PriceTag,
    #[token("\"q\"")] QtyTag,
    #[token("\"T\"")] TradeTimeTag,
    #[token("\"m\"")] BuyerMakerTag,
    #[token("\"stream\"")] StreamTag,
    #[regex(r#"\w+@\w+"#)] StreamName,
    #[regex(r#"(true|false)"#)] Bool,
    #[regex(r#"\"[+-]?((\d+\.?\d*)|(\.\d+))\""#)] QuotedNumber,
    #[regex(r"[-+]?\d+")] Integer,
    #[regex(r#"\"[a-zA-Z0-9]+\"#, priority = 1)] Text,
}

Key names are compiled into the DFA, so matching "p" becomes an O(1) state transition instead of a runtime string comparison.

Step 2 – Stream Scan + Borrowed Slices

#[derive(Default, Debug)]
struct TradeUpdate {
    symbol_off: (usize, usize), // offset range in the source buffer
    price: f64,
    qty: f64,
    agg_id: u64,
    event_time_ns: i64,
    trade_time_ns: i64,
    is_buyer_maker: bool,
}

fn parse_agg_trade(msg: &str) -> Result<TradeUpdate, ParseError> {
    let mut lex = AggTrade::lexer(msg);
    let mut t = TradeUpdate::default();
    while let Some(tok) = lex.next() {
        match tok {
            Ok(AggTrade::SymbolTag) => {
                if lex.next().is_some() {
                    let s = lex.slice(); // e.g. "SOLUSDT"
                    let inner = &s[1..s.len()-1]; // strip quotes, still borrowed
                    let start = inner.as_ptr() as usize - msg.as_ptr() as usize;
                    t.symbol_off = (start, start + inner.len());
                }
            }
            Ok(AggTrade::PriceTag) => {
                if lex.next().is_some() {
                    let s = lex.slice();
                    t.price = fast_float2::parse(&s[1..s.len()-1])?;
                }
            }
            Ok(AggTrade::QtyTag) => {
                if lex.next().is_some() {
                    let s = lex.slice();
                    t.qty = fast_float2::parse(&s[1..s.len()-1])?;
                }
            }
            Ok(AggTrade::AggIdTag) => {
                if lex.next().is_some() {
                    t.agg_id = lex.slice().parse()?;
                }
            }
            Ok(AggTrade::EventTimeTag) => {
                if lex.next().is_some() {
                    let ms: i64 = lex.slice().parse()?;
                    t.event_time_ns = ms * 1_000_000;
                }
            }
            Ok(AggTrade::TradeTimeTag) => {
                if lex.next().is_some() {
                    let ms: i64 = lex.slice().parse()?;
                    t.trade_time_ns = ms * 1_000_000;
                }
            }
            Ok(AggTrade::BuyerMakerTag) => {
                if lex.next().is_some() {
                    t.is_buyer_maker = lex.slice() == "true";
                }
            }
            _ => {}
        }
    }
    Ok(t)
}

Observations: lex.slice() always returns a borrowed view; values never leave the original buffer.

Key comparison is a DFA transition, not a runtime string equality.

Numeric fields are parsed in‑place with fast_float2, avoiding intermediate String allocations.

The symbol can be stored as a (start, end) offset, postponing any to_owned() until it is truly needed.

4. Other Zero‑Copy Parsing Options in the Rust Ecosystem

serde + borrowed deserialization : Supports &'a str and &'a [u8] via #[serde(borrow)]. Fast for configuration or non‑hot paths, but still incurs generic JSON parsing overhead.

simd‑json / sonic‑rs : SIMD‑accelerated JSON parsing, 2‑3× faster than serde_json. Requires a mutable input buffer and specific CPU instruction sets.

nom / winnow : Parser combinators that work on &[u8] or &str and naturally return borrowed slices. Very expressive for custom binary/text protocols, but have a steeper learning curve.

bytes + memchr : Low‑level building blocks; bytes::Bytes gives reference‑counted zero‑copy buffers, while memchr provides SIMD‑fast byte searching.

rkyv / zerocopy : Zero‑copy deserialization for binary layouts, turning &[u8] directly into struct references ("zero‑parse"). Not applicable to text protocols but useful for shared‑memory or replay data.

Selection Summary

Known‑schema market JSON, ultra‑hot path → logos (specialised DFA) or sonic‑rs (lazy parsing).

Complex / mutable JSON needing generality → simd‑json / sonic‑rs .

Configuration or non‑hot path → serde_json (simple and reliable).

Custom binary/text protocol (FIX, proprietary TCP) → winnow / nom .

Hand‑written extreme parser foundation → bytes + memchr .

Binary layout, shared memory, replay data → rkyv / zerocopy .

5. Implementation Checklist

Prefer borrowed types ( &'source str or &'source [u8]) for hot‑path token fields; avoid String.

Encode schema knowledge into the DFA: fixed keys become literal tokens, turning key comparison into a state transition.

Parse numbers in‑place with zero‑copy float parsers (e.g., fast-float2 or lexical).

Bind the lifetime of parsed results to the source buffer; only call .to_owned() when data must outlive the buffer.

For HFT keep the default unsafe implementation of logos (justified by the SAFETY comment); switch to forbid_unsafe only when compliance demands it.

Select a single library for each layer: sonic‑rs for JSON throughput, winnow for custom protocols, bytes for zero‑copy buffer handling.

6. Conclusion

The parser only holds references and offsets; slice() directly borrows the source buffer; fixed‑size chunks batch reads to amortise boundary checks; and token rules are compiled into a DFA at build time. This design aligns with HFT requirements of zero allocations, low jitter, and a hot path free of superfluous instructions.