How Baidu Transformed E‑commerce Risk Control with Multi‑Modal AI Agents

Background and Problems

Baidu Preferred, the e‑commerce brand under Baidu, faces a three‑cornered challenge of safety, efficiency, and user experience in its risk‑control system. Traditional machine‑review suffers from weak multimodal detection, ambiguous semantics, and poor audit experience, leading to slow manual reviews, vague rejection reasons, and potential regulatory or reputation risks.

Technical Solution

The proposed solution integrates a large multimodal model , deterministic rule engine , and a domain‑specific knowledge base into a collaborative machine‑review agent.

Overall Architecture

Standardization Layer : Consolidate 700+ scattered risk points into 24 core risk groups covering >95% of violations.

Feature Extraction Layer : Combine rule‑based extraction, LLM text understanding, and multimodal image understanding to capture textual, visual, and structured data.

Risk Judgment Layer : Use rules for deterministic checks, knowledge‑base queries for external verification, and LLMs for comprehensive decision making.

Output Layer : Provide clear pass/fail results, natural‑language rejection reasons, and actionable remediation suggestions.

Key Steps

Risk Point梳理 (Reduction) : Collect Q1 2025 human‑review records, merge similar risks via hierarchical clustering, and prune low‑severity or low‑frequency risks.

Standard Optimization (Normalization) : Classify risks into “definite violation”, “definite non‑violation”, and “boundary” categories.

Dynamic Update (Iteration) : Monthly patrols and appeal data continuously enrich risk groups and update the agent.

Practical Cases

3.1 Qualification‑Missing Risk

High‑risk categories (e.g., health products, medical devices) require complete qualification documents. Traditional rules only check keyword presence, missing the link to qualification databases. The AI solution fine‑tunes a domain model using high‑quality samples, prompt engineering, data augmentation, and full‑parameter or instruction tuning to accurately detect missing qualifications.

3.2 Brand‑Authorization Risk

Detecting counterfeit branding involves multimodal logo recognition, LLM‑based textual similarity, and knowledge‑base checks against authorized brand lists. The pipeline extracts logo features from product images, matches them with brand‑authorization records, and generates precise rejection messages such as “Missing Nike authorization certificate”.

3.3 Category‑Misplacement Risk

Incorrect category placement harms traffic and compliance. The solution evolves from a simple similarity‑based recommendation to a two‑stage recall‑and‑rank system using bge‑large‑emb embeddings, and finally adds targeted recall paths and a combined LLM‑rule judgment to improve precision for high‑risk items.

Results and Impact

The deployment achieved a “three‑rise, three‑drop” effect:

Rise: higher machine‑review coverage, faster audit latency, improved merchant satisfaction.

Drop: reduced manual review volume, lower appeal rate, fewer user complaints.

Experience Summary

Multimodal Fusion : 80% of e‑commerce data is multimodal; single‑modal models cannot cover all scenarios.

Explainability First : LLM‑generated natural‑language feedback dramatically improves merchant remediation.

Closed‑Loop Iteration : Continuous data‑driven model updates create a virtuous flywheel of performance gains.

Outlook

Future work will focus on self‑optimizing agents that generate hard samples for continual learning, further expanding AI‑driven risk control across the e‑commerce ecosystem.

Images