Multi-turn Dialogue Intent Classification: Data Processing, Model Construction, and Operational Optimization

Multi‑turn dialogue chatbots are a typical application of artificial intelligence, but building them is challenging because it requires heterogeneous knowledge representation, extraction, reasoning, and the integration of core AI technologies such as natural language understanding.

The article focuses on the second stage of a dialogue system—topic intent recognition. In the described game‑security scenario, seven hierarchical topic categories are defined (security mode, stolen, mistake, ban, credit, report, and common). Topic intent classification is treated as a text‑classification task.

The data‑processing pipeline consists of extracting classic utterances (using topic modeling, sentence‑vector clustering, graph‑based PageRank, etc.), manually labeling them, and then automatically expanding the label set by measuring text similarity (edit distance, weighted word vectors, Seq2Seq‑AutoEncoder, BERT sentence vectors) and filtering bad cases.

For model construction, the workflow includes:

Data cleaning with regex and phrase‑structure rules.

New‑word discovery (adding game‑specific nouns to Jieba, replacing them with placeholders such as GameName or AreaName ).

Pre‑training Word2Vec on the full corpus to obtain word and POS embeddings.

Training a two‑layer CNN combined with a Highway network; the model predicts multi‑label probabilities and uses a 0.3 threshold to assign the common class when all probabilities are low.

The final CNN‑based topic classification model achieves high performance on a held‑out test set (27521 samples):

Accuracy

Recall

F1_score

0.986

0.982

0.984

Hot‑word analysis for each topic is performed either with traditional TF‑IDF/ TextRank methods or by extracting attention weights from an RNN‑Attention model.

The self‑attention formula (multi‑head) is illustrated below:

Operational optimization focuses on analyzing bad cases. Reasons include overlapping vocabulary across topics, mislabeled training data, and unseen samples. Corrections are applied at three levels:

Data layer – re‑labeling and cleaning samples (diagram below).

Model layer – balancing latency and accuracy (diagram below).

Rule layer – adding topic‑specific regex filters to capture hard‑to‑learn cases.

To scale the dataset, an ensemble‑model approach is used for automatic label expansion. Starting from ~9k seed samples, automatic expansion yields >70k labeled instances, which are then used to train four different neural classifiers. The best model (a multi‑head self‑attention + Highway network) reaches 98.7% accuracy on the full training set.

Key practical recommendations include:

Pre‑train Word2Vec on the full corpus or combine character‑ and word‑level embeddings to reduce OOV issues.

Use a CNN baseline with batch‑norm and Highway layers for fast iteration.

Handle class imbalance via oversampling, undersampling, threshold tuning, focal loss, or data augmentation (shuffle, synonym replacement, template modification, GAN‑based generation).

Adopt multi‑label sigmoid loss when topics are not mutually exclusive.

For small datasets, fine‑tune BERT or employ few‑shot learning techniques such as Induction Networks.