Baidu Mini Program Online Quality Assurance System: AI‑Driven Automated Traversal, Page Anomaly Detection, and Cloud Phone Cluster

To discover red‑line issues across all Baidu mini‑programs, protect the online ecosystem, improve user experience, and increase audit efficiency, Baidu Mini Program QA built an online quality‑assurance system that heavily leverages AI technologies.

The system focuses on three core capabilities: (1) automated traversal of mini‑programs, (2) page‑anomaly detection, and (3) a cloud‑phone cluster that provides scalable device resources.

Overall background : Baidu hosts hundreds of thousands of mini‑programs running on dozens of open‑source host apps, each with its own distribution scenario. QA must ensure quality both internally (framework stability, smart delivery) and externally (online red‑line issues, rights‑level support).

Core capabilities are realized through:

Automated traversal ability – acquiring necessary runtime information via various traversal algorithms.

Page‑anomaly detection – automatically checking collected information for violations.

Clustered device resources – solving the large‑scale parallel inspection problem.

Automated traversal development : Baidu built the betterAutoTest engine, consisting of Bat Engine (core code injected into the runtime), Bat Agent (WebSocket server on PC), and Bat Driver (NodeJS client library). It supports four device types (real phone, development board, cloud phone, web‑based) across all host apps, achieving ~90% coverage, ~100 ms per command latency, and 99.9% stability.

Exploration stages progressed from a monkey‑style random click strategy, to a behavior‑prediction approach based on historical click data, and finally to a target‑recognition method that uses image segmentation, OCR, object detection, color analysis, element attribute judgment, aggregation, block division, and generation of a page‑structure‑tree JSON.

Target‑recognition workflow includes:

Image slicing to locate element boundaries.

OCR to extract text and coordinates.

Object detection to identify icons.

Color analysis to filter noisy or text‑heavy regions.

Element attribute determination and aggregation.

Block division and page‑structure‑tree generation.

Using this tree, Baidu can recognize common controls such as article lists, product cards, and bottom navigation tabs.

Deep‑learning supplement : A YOLO‑v3 model was trained on automatically labeled data (plus manually corrected bad cases) to improve recall for target‑recognition tasks.

Page anomaly detection combines deep‑CNN feature extraction for screenshot similarity with the page‑structure‑tree for positional checks, covering screenshots, text, DOM, and source code.

White‑screen detection addresses full/partial/skeleton white screens, long‑loading screens, and partial image‑load failures using image segmentation, color analysis, DOM parsing, and structure‑tree‑based image‑failure identification.

Scenario‑aware detection reduces false positives on multi‑level pages by considering page context (e.g., navigation path) and historical thresholds.

Cloud‑phone cluster construction evolved through three generations: 1.0 – many physical phones with manual ops; 2.0 – unified development boards with semi‑automatic ops; 3.0 – Baidu Cloud‑phone service with fully automated orchestration, dramatically lowering hardware and O&M costs.

Business impact : The platform handles >200 k daily device tasks with >99.3% success, has recalled >100 k red‑line issues, intervened in >80 k mini‑programs, and auto‑detects >82% of issues without human intervention. It also supports dozens of promotional campaigns with >85% issue‑recall rates.