How Alibaba Uses AI to Automatically Find High‑Quality Buyer Photos on Taobao

Background

Taobao’s buyer‑generated content (UGC) varies widely in quality, making manual selection of high‑quality images impractical for operation staff. Strict visual standards—clear lighting, harmonious composition, no ads—lead to a low pass rate (<30%). An automated solution is therefore essential.

Overall Solution

The proposed pipeline first defines three UGC categories: approved UGC (high‑quality), highlighted UGC (merchant‑approved), and ordinary UGC (the rest). The goal is to surface a diverse set of high‑quality UGC.

UGC Quality Evaluation Model

Operational reviewers assess images and text, turning the problem into a classification task.

1. Feature Selection

Statistical features from user, product, and feedback data are extracted (see table in original). A Gradient Boosted Decision Tree (GBDT) model predicts UGC quality, confirming the feasibility of a classification approach.

2. Classification Formulation

Initially a binary label (1 = approved, 0 = rejected) was used, but a three‑class scheme (2 = approved, 1 = merchant‑highlighted, 0 = ordinary) better matches real‑world distribution and yields higher performance.

3. Image Semantic Features

A CNN (ResNet‑50 fine‑tuned from ImageNet) evaluates visual quality, boosting approval rates by over 100% compared with the previous pipeline.

4. Aesthetic Features

The AVA database provides style labels and aesthetic scores. A Brain‑Inspired Deep Network learns aesthetic representations, which are combined with ResNet semantic features and statistical embeddings to predict UGC quality. This addition improves accuracy, recall, and F1 on the validation set, and AB testing shows a >6% increase in approval rate.

Dirty Data Handling

Several types of noisy UGC are identified and filtered:

Comment Sentiment : Attn‑BiLSTM outperforms TextCNN for detecting negative reviews, achieving F1 > 0.9 on a four‑grade sentiment dataset.

N‑Gram Filtering : Repeated template comments are removed by detecting common 2‑, 3‑, and 4‑grams combined with length and entropy checks.

OCR & Image Hash : OCR extracts text from images, while perceptual hashing discards duplicated or stolen images.

Irrelevant Image Detection

After initial filters, 10‑15% of images remain unrelated (e.g., scenery, memes). A binary classifier trained with negative samples (stolen, meme, web images) and positive samples (approved UGC) uses ResNet features, category embeddings, and user behavior metrics to flag irrelevant images.

Key Takeaways

High‑quality data outweighs sophisticated features; realistic data dramatically improves model performance.

When data is scarce, quickly labeling a few thousand examples can yield surprisingly strong results.

Fine‑tuning ImageNet‑pretrained models on small datasets often outperforms training from scratch.

Data augmentation (flipping, rotation, random cropping) enhances model generalization.