Chinese Text Sentiment Classification Using Multi‑layer LSTM: Data Preparation, Model Architecture, and Business Applications

Introduction

Deep learning (deep neural networks) as an important branch of machine learning continuously drives progress in many fields, including sentiment classification for text processing. By encoding text more effectively, deep‑learning‑based sentiment classification can achieve higher accuracy than traditional shallow machine‑learning or statistical methods. This article shares the practice of Chinese text sentiment classification in Tencent Goose Man U‑Product, illustrating the whole workflow and discussing challenges.

Development and Challenges of Text Sentiment Analysis

Sentiment Analysis Development

Sentiment analysis (also called sentiment classification) belongs to the Natural Language Processing (NLP) field. It usually determines whether a piece of text expresses a positive, negative, or neutral attitude, and some research further grades sentiment intensity.

Before 2000, limited Internet data meant little research on this task. After 2000, the rapid growth of online text made sentiment analysis a hot topic, initially focusing on English. Pang, Lee and Vaithyanathan (2002) first applied Naïve Bayes, Maximum Entropy, and SVM to movie reviews. From 2000‑2010, sentiment analysis relied mainly on traditional statistical and shallow machine‑learning methods.

Since 2010, with the rise of deep learning, sentiment analysis has gradually shifted to deep‑learning‑based approaches, which have consistently outperformed traditional methods.

Challenges Specific to Chinese Text Sentiment Analysis

Chinese presents several difficulties:

Word segmentation errors: Chinese characters form words without explicit delimiters, making accurate segmentation challenging.

Lack of comprehensive sentiment lexicons: Unlike English, Chinese lacks a complete sentiment‑word database, and new slang (e.g., “666”) constantly emerges.

Negation handling: Sentences like “我不是很喜欢这个商品” vs. “我很喜欢这个商品” contain the same sentiment word but opposite meanings due to negation.

Domain‑specific polarity: Some neutral words become sentiment‑bearing in specific contexts (e.g., “蓝屏” is neutral but negative in a phone‑review scenario).

These challenges affect both traditional machine‑learning and deep‑learning methods, although deep learning can alleviate some of them.

Deep Learning Overview

Deep learning, first proposed by Hinton, Bengio and others in 2006, is a major branch of machine learning inspired by the brain’s neuronal structure. Neurons consist of dendrites (inputs) and axons (outputs); connections (synapses) transmit signals and can be strengthened or weakened.

The human brain contains roughly 100 billion neurons and 1 trillion synapses, forming a massive network that inspired the concept of “deep/multi‑layer neural networks.”

Although introduced in 2006, deep learning gained industrial attention after the 2012 ImageNet competition, where it boosted image‑classification accuracy from 74 % to 84 %. Subsequent research pushed the accuracy to 97.3 % (2017), surpassing human performance.

Beyond image recognition, deep learning has dramatically improved speech recognition, machine translation, and other tasks, leading to high‑profile events such as AlphaGo’s 2016 victory, which propelled AI into the public eye.

Chinese Word Segmentation Overview

Effective Chinese sentiment classification relies on accurate word segmentation because Chinese lacks spaces between words. Good segmentation is a prerequisite for downstream NLP tasks.

Dictionary‑Based Segmentation Methods

These methods build a lexicon and match text against it. They are fast and controllable, widely used in industry. Early algorithms include Forward Maximum Matching (FMM) and Reverse Maximum Matching (RMM); RMM generally yields better results. Later, Bi‑directional Matching (BM) and Optimum Matching (OM) combine both directions and word frequency.

FMM: 结婚/的/和尚/未/结婚/的 (incorrect) RMM: 结婚/的/和/尚未/结婚/的 (correct)

Further improvements such as BM and OM aim to increase segmentation accuracy.

Statistical (Dictionary‑Free) Segmentation Methods

These approaches treat segmentation as an unsupervised learning problem, constructing an implicit lexicon from large corpora. Techniques include mutual information, conditional entropy, N‑gram models, Hidden Markov Models (HMM), Maximum Entropy (ME), and Conditional Random Fields (CRF). They excel at discovering out‑of‑vocabulary words but may suffer from efficiency and consistency issues.

Multi‑layer LSTM Chinese Sentiment Classification Model

After segmentation, the sentiment classifier is a supervised learning task using a multi‑layer LSTM network. Input: a segmented Chinese sentence; Output: a probability distribution over positive and negative classes.

Data Preparation

We collected over 400 k real Goose Man user comments and sampled 70 k balanced comments (positive/negative) for training. Text is first tokenized, then encoded. One‑Hot encoding is simple but memory‑intensive (≈38 k dimensions for our vocabulary). Therefore we adopt word embeddings (128‑dim vectors) to reduce memory and computation.

Example of One‑Hot encoding (5‑word vocabulary) is shown in the figure. Word‑embedding visualization for a 2‑dim example is also provided.

Model Building

The implementation uses Keras on top of TensorFlow. The model consists of six layers: an Embedding input layer, two LSTM layers, a Flatten layer, a Dense layer, and a Softmax activation.

EMBEDDING_SIZE = 128
HIDDEN_LAYER_SIZE = 64

model = Sequential()
model.add(layers.Embedding(words_num, EMBEDDING_SIZE, input_length=input_data_X_size))
model.add(layers.LSTM(HIDDEN_LAYER_SIZE, dropout=0.1, return_sequences=True))
model.add(layers.LSTM(64, return_sequences=True))
# model.add(layers.Dropout(0.1))
model.add(layers.Flatten())
model.add(layers.Dense(2))  # [0,1] or [1,0]
model.add(layers.Activation('softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.summary()
model.fit(X, Y, epochs=1, batch_size=64, validation_split=0.05, verbose=2)

The Flatten layer compresses the 2‑D tensor (20 × 64 = 1280) into a 1‑D vector; the Dense layer maps it to two output neurons representing positive and negative probabilities.

Model Training

Training on a 8‑core CPU with 8 GB RAM takes about 3 minutes for one epoch over 70 k samples. The trained model achieves ~96 % accuracy on the test set, compared to 75‑90 % for traditional machine‑learning baselines. The model is specialized for Goose Man comment vocabulary; performance may drop on out‑of‑domain texts.

Business Application Scenarios and Future Outlook

Application Scenarios

In Goose Man U‑Product, users leave comments after purchase. Some “fake positive” reviews are actually negative (≈3 % of all comments). Manual handling is costly; the sentiment classifier can automatically detect such cases.

Another scenario is extracting “deep positive” reviews: comments with high positive sentiment scores and longer length are highlighted on product pages, improving user experience and reducing copy‑writing workload. Conversely, “deep negative” reviews help operations understand pain points.

Tencent AI Lab also provides a character‑level sentiment API, which we combine with our word‑level model for higher quality analysis.

Future Extensions

Beyond binary sentiment, we plan to perform aspect‑based sentiment analysis and dependency parsing to extract key user opinions. This would enable comprehensive product opinion summarization, guiding product improvement and operational decisions.

References

• TensorFlow (Google): https://tensorflow.google.cn/ • Keras (high‑level TensorFlow API): https://keras.io/