Boost NLP Model Performance with n-gram Feature Engineering

1 Significance of Feature Engineering

In NLP tasks, raw token sequences mapped to vectors often miss deep semantic cues. Feature engineering addresses three needs:

Semantic completion – capture phrase‑level dependencies beyond single words.

Model adaptation – construct matrix inputs that satisfy algorithmic requirements.

Metric improvement – richer features directly raise accuracy, recall, etc.

Example: In e‑commerce review sentiment analysis, word‑frequency treats “this phone is terribly bad” and “this phone is bad terribly” identically, while bi‑gram distinguishes the differing phrases.

2 n‑gram Feature Enhancement

2.1 Context Feature Capture

The n‑gram model slides a window of size n over the token sequence and treats each consecutive group as a combined feature.

Technical Evolution

bi‑gram (n=2) – captures short collocations such as “流量套餐” vs “套餐推荐”.

tri‑gram (n=3) – identifies brief patterns like the positive phrase “送货速度快”.

higher‑order (n≥4) – useful for domain‑specific terms but can cause dimensional explosion.

Warning: In customer‑service dialogue, a 5‑gram may enlarge the feature space by ~100×; combine with TF‑IDF or other selection methods.

2.2 Algorithm Implementation

def generate_ngram_features(token_ids, n=2):
    """Build a context feature enhancement engine.
    :param token_ids: list of token IDs, e.g., [142, 29, 87]
    :param n: context window size
    :return: set of n‑gram feature tuples
    """
    return set(zip(*[token_ids[i:] for i in range(n)]))

Practical example :

comment_tokens = [15, 239, 76, 89]  # corresponds to "快递 服务 非常 差"
ngrams = generate_ngram_features(comment_tokens, n=2)
print(ngrams)  # {(15, 239), (239, 76), (76, 89)}

3 Text Dimension Standardization

3.1 Necessity of Length Normalization

Deep‑learning models require uniform tensor shapes for three reasons:

Compute resource optimization – GPUs need consistent matrix dimensions.

Model architecture constraints – RNNs/LSTMs expect a predefined time‑step length.

Information density balance – avoid noise from overly long texts and loss from overly short ones.

Analysis of an e‑commerce dataset shows 90 % of comments are 15–50 characters long; setting cutlen=40 covers most cases while allowing intelligent truncation.

3.2 Dynamic Padding Implementation

from keras.preprocessing.sequence import pad_sequences

def dynamic_padding(text_matrix, maxlen=40, padding='post', truncating='pre'):
    """Intelligent text dimension calibrator.
    :param text_matrix: original text matrix
    :param maxlen: maximum retained length
    :param padding: 'post' pads after the sequence
    :param truncating: 'pre' cuts from the beginning
    :return: standardized text matrix
    """
    return pad_sequences(text_matrix, maxlen=maxlen,
                         padding=padding, truncating=truncating)

Strategy recommendations :

Product titles – keep trailing keywords (post‑truncating).

News articles – preserve leading lead (pre‑truncating).

Dialogue scenes – apply sliding‑window extraction of core segments.

4 Deployment Recommendations

Feature dimension control – with a vocabulary of ~20 k, keep bi‑gram features under ~50 k.

Dynamic length per business line – set different cutlen values according to use case.

Hybrid feature engineering – combine n‑gram with character‑level features for richer representation.

Monitoring feedback loop – maintain a feature‑importance evaluation system and iteratively refine the feature set.

The full source code and examples are available in the GitHub repository https://github.com/JavaEdge/Java-Interview-Tutorial.