Transforming Regular Expressions into Neural Networks for Text Classification and Slot Filling

01 Regular Expression Introduction

Regular expressions (regex) are widely used pattern‑matching rules in many deployed systems. They are simple, interpretable, support fine‑grained diagnostics, and require no training data, making them ideal for cold‑start scenarios. However, regexes need expert authors, may have low recall, and cannot be improved with labeled data.

Motivation

To combine the interpretability of regexes with the learning ability of neural networks, we transform a regex into an equivalent neural network that can be constructed without data and later fine‑tuned when data become available.

02 Converting Regex to a Neural Network

1. Regex → Finite‑State Automaton (FSA) : Using classic algorithms, a regex is converted into an equivalent FSA with start, final, and intermediate states.

2. FSA as a Recurrent Neural Network : The FSA is represented by three binary parameters—transition tensor T , start vector α , and final vector β . Forward scoring is performed by multiplying the start vector, a sequence of transition matrices (derived from T ), and the final vector, which mirrors the recurrent step of an RNN.

3. Tensor Decomposition : The transition tensor is factorised via rank decomposition into three matrices (word embedding, state‑to‑state matrices), reducing parameter count to that of a standard RNN.

4. Integrating Pre‑trained Word Embeddings : External embeddings (e.g., GloVe, BERT) are combined with the decomposed embeddings, allowing the model to benefit from lexical knowledge while preserving the original regex‑derived parameters.

The resulting model, named FA‑RNN , is fully constructed from the regex (no training required) but can be further trained to improve performance.

03 Text Classification with FA‑RNN

In the original regex system, each rule yields a binary match vector that is aggregated by a logical layer to produce a single label. FA‑RNN replaces this pipeline: the network outputs real‑valued scores (≈0/1) for each rule, which are fed into a two‑layer MLP that mimics the logical layer. Experiments show that FA‑RNN matches the zero‑shot performance of the regex system and outperforms neural baselines in low‑resource settings, while achieving comparable results when abundant data are available.

04 Restoring Regex from a Trained FA‑RNN

After training, the three decomposed matrices are multiplied to reconstruct the transition tensor, which is then binarised to obtain a new FSA and consequently a regenerated regex. The regenerated regex can improve over the original in some datasets.

05 Slot Filling with Regex‑Based FST‑RNN

For slot‑filling tasks, regexes with capturing groups are converted into finite‑state transducers (FST) that output BIO tags. The FST is represented by a transition tensor, start vector, and final vector, similar to the FA‑RNN case but with an extra output dimension.

1. FST as a Bi‑RNN : Forward and backward passes compute α and β vectors for each position; their combination yields label distributions, analogous to a bidirectional RNN.

2. Independent FST (iFST) : By assuming conditional independence between states, the four‑order tensor is split into a three‑order tensor and a matrix, reducing complexity.

3. Tensor Decomposition & Pre‑trained Embeddings : Rank decomposition lowers the iFST to standard RNN complexity, and external embeddings are blended with the learned embeddings, forming the FST‑RNN model.

Experiments on slot‑filling demonstrate that FST‑RNN inherits the zero‑shot equivalence to the original regex and outperforms neural baselines in low‑resource scenarios, matching them when data are plentiful.

06 Summary

We presented methods to convert regular expressions into neural networks: FA‑RNN for classification and FST‑RNN for slot filling. These models retain the interpretability and cold‑start capability of symbolic systems while allowing data‑driven fine‑tuning, achieving strong performance across resource regimes.

Thank you for listening.