NLP Techniques for Classifying Ctrip Ticket Customer Service Conversations

Ctrip places great emphasis on efficient and satisfying customer service throughout the entire travel booking lifecycle, routing users first to an intelligent chatbot and then to human agents when needed; the resulting conversation logs are classified using NLP to guide operational decisions.

The classification task is a Chinese text‑classification problem. Traditional statistical models, word‑embedding‑based deep neural networks, and modern pretrained language models (e.g., Word2Vec, attention mechanisms, BERT) are reviewed, highlighting their evolution and relevance to the task.

Data preprocessing addresses Chinese‑specific challenges: removal of fixed service phrases via regex, conversion of traditional to simplified characters, tokenization with jieba or HanLP, synonym replacement using a domain‑specific dictionary, filtering of punctuation, numbers and emojis, and length normalization by truncating or padding to the 95th‑percentile length.

Modeling starts with a Bi‑GRU baseline (78.12% accuracy). Improvements include adding self‑attention (Bi‑GRU+Self‑Attention, 80.13%), hierarchical attention networks (HAN, 80.97%), and fine‑tuned BERT (82.84%). Bad‑case analysis reveals three error sources: insufficient word‑importance modeling, poor recognition of industry terms, and missing contextual features. Optimizations combine self‑attention with Bi‑GRU, incorporate industry vocabularies, and embed contextual scene features, culminating in an improved Bi‑GRU+Self‑Attention model that reaches 84.47% accuracy.

The study demonstrates a stepwise accuracy gain from classic statistical methods to advanced pretrained models and contextual enhancements, and suggests future work integrating pretrained language models with richer context information for further performance improvements.