Risk‑Driven Delivery and Quality Assessment Model for Automated Testing at Baidu

Based on risk‑driven delivery, Baidu explores intelligent testing in the perception‑intelligence stage, identifying three realities: most projects have no bugs, many test tasks cannot uncover errors, and testers may miss defects.

The series will publish three articles, the third of which focuses on a quality‑assessment model that helps improve risk‑decision levels.

Background : Manual testing decisions rely on human judgment, leading to inconsistent quality and efficiency. The article proposes using machine‑learning models to automate or assist decision‑making.

Manual decision steps include reviewing delivery data, giving a decision conclusion, and following up on bug leakage. Drawbacks are high cost of data collection, reliance on uneven human experience, and loss of expertise when staff turnover occurs.

Machine‑learning can replace human decisions in scenarios such as autonomous driving, self‑inspection systems, and facial security checks, suggesting a similar approach for testing risk decisions.

The overall solution consists of three parts: risk identification (collecting dynamic and static risk points), risk control (recommending test activities and generating inputs), and risk decision (evaluating residual risk probability and impact to produce test suggestions, risk levels, and conclusions).

Risk identification gathers five dimensions with over 50 features, linking test tickets, requirement cards, and pipeline IDs to build feature lineage, while allowing custom feature retrieval.

Risk Control focuses on determining which tests to run. Traditional exhaustive testing wastes resources, whereas risk‑driven execution runs only necessary tests, improving efficiency and reducing blind spots.

Risk Decision combines rule‑based and model‑based assessments. Examples include medical check‑ups and credit risk, where rules filter obvious cases and models (e.g., logistic regression) estimate probabilities for ambiguous cases.

The logistic regression model is chosen for its interpretability and suitability for limited data; the formula is shown in the accompanying image.

Risk matrix visualization maps probability (y‑axis) against impact (x‑axis) to categorize outcomes (e.g., high‑risk interception, low‑risk flow‑through).

Risk visualization reports present risk data and decision recommendations, allowing QA feedback to close the loop and continuously improve the model.

Deployment results (Q3 2022) show significant quality and efficiency gains: 1,123 non‑autonomous projects identified with 318 bugs intercepted; 4,345 autonomous projects saved 2,172 person‑days, and evaluation waiting time reduced from 50 h to 2 h.

The current stage corresponds to assisted decision (similar to L2/L3 in autonomous driving). Future work aims to reach conditional and high automation, ultimately achieving fully automated decision‑making.

In summary, the quality‑risk assessment system comprises risk identification, control, decision, and feedback loops, and is progressing toward fully automated intelligent delivery.