Multi-Task Learning for Category Prediction in Zhaozhuan Search Intent Understanding

1 Multi‑Task Learning Overview

1.1 What Is Multi‑Task Learning

Multi‑task learning (MTL) is a machine‑learning paradigm that leverages information from several related tasks to improve the generalisation performance of all tasks by sharing representations and enabling knowledge transfer.

MTL differs from related paradigms such as transfer learning, multi‑label learning, multi‑output regression, and multi‑view learning in how tasks and data are organised and how shared knowledge is exploited.

1.2 Multi‑Task Learning in NLP

Recent advances in NLP, driven by transformer‑based pre‑trained language models (e.g., BERT, T5, GPT‑3), have achieved strong results on many downstream tasks, but fine‑tuning these models often requires large labelled datasets. Combining pre‑trained language models with MTL helps alleviate data scarcity and improve cost‑effectiveness.

MTL architectures in NLP can be classified into parallel, hierarchical, modular, and generative‑adversarial designs, each handling task relationships differently.

2 Industry Category‑Prediction Methods

Intent understanding in e‑commerce focuses on narrow‑scope category prediction (CP), aiming to identify the intent category of a query.

Alibaba introduced a deep hierarchical classification framework that incorporates multi‑scale hierarchical information and a joint loss to penalise hierarchical prediction errors.

Meituan splits intent recognition into recall (binary classification) and distribution (ranking) stages, using dictionary‑based rules for frequent intents and BERT for long‑tail cases, with features from CTR, CVR, and embeddings.

JD.com employs hierarchical multi‑label classification and semantic models that fuse click, order, and transaction features to score categories.

3 Multi‑Task Learning Practice at Zhaozhuan

Zhaozhuan aims to enhance search experience across all product categories while maintaining performance in the mobile 3C domain.

3.1 Practical Application in Category Prediction

The platform predicts three hierarchical levels: category, brand, and model. Data challenges include imbalance, inconsistent hierarchy, and a massive label space (thousands of brands, tens of thousands of models). To address these, Zhaozhuan applies down‑sampling of dominant categories, up‑sampling of minority categories, and treats category and brand prediction as classification tasks while handling model prediction via text matching.

For classification, RoBERTa encodes the query and a fully‑connected layer outputs class scores. For model matching, a RoBERTa‑based Siamese network generates embeddings for the query and candidate model, and similarity is computed. The overall loss is a weighted sum of the three task losses (cross‑entropy for classification, SimCSE contrastive loss for matching).

Multi‑task learning improves model performance, generalisation, data efficiency, and enables knowledge transfer, leading to higher conversion rates in online experiments.

3.2 Future Plans

Human evaluation and online A/B tests confirm the effectiveness of MTL. Future work includes extending MTL to other intent‑understanding modules such as named‑entity recognition and applying shared textual features to recall modules (vector‑based or Elasticsearch‑based).

References

[1] A Survey on Multi‑Task Learning.

[2] Multi‑Task Deep Neural Networks for Natural Language Understanding.

[3] Multi‑Task Learning in Natural Language Processing: An Overview.

[4] Deep Hierarchical Classification for Category Prediction in E‑commerce System.

[5] https://zhuanlan.zhihu.com/p/370576330

[6] https://www.modb.pro/db/152185

[7] RoBERTa: A Robustly Optimized BERT Pretraining Approach.

[8] SimCSE: Simple Contrastive Learning of Sentence Embeddings.