Optimizing Regular Expression Engines for High‑Performance Deep Packet Inspection

Regular expressions are fundamental for string processing and deep packet inspection, but traditional engines struggle with large or numerous patterns due to high memory consumption and slow matching.

The article introduces a high‑efficiency NFA construction method that bypasses syntax‑tree creation, reducing both construction time and memory usage.

It then describes an NFA engine with fewer empty jumps, optimizing connection, OR, and closure operations to produce smaller, faster automata.

Prefix and suffix optimizations are applied to remove unnecessary ".*" patterns in substring‑search mode, further accelerating NFA‑to‑DFA conversion.

A fast NFA‑to‑DFA conversion algorithm uses radix‑tree lookup and ordered edge lists to minimize lookup overhead and improve transition handling.

Space‑compression techniques for both NFA and DFA store transitions as single characters or character ranges, dramatically reducing memory footprints.

A novel hybrid finite automaton combines partially built DFA with tail‑NFA to balance performance and resource constraints.

Finally, a large‑scale regex matching algorithm extracts fixed‑length fingerprints, builds a memory‑controlled Aho‑Corasick filter, and performs ordered fingerprint comparison to efficiently filter and verify massive regex sets.

References to prior work on NFA ε‑jump optimization, automaton compression, parallel matching, and hybrid automata are provided.