How Risk‑Driven Delivery Boosts Test Efficiency with AI‑Powered Quality Models

Background and Motivation

In many software projects, over 80% have no associated bugs or production issues, yet testing still relies on manual judgments that vary by individual skill, leading to inefficiencies, missed defects, and knowledge loss.

The authors observed three realities: (1) most projects lack bugs, (2) many test tasks fail to uncover defects, and (3) testers can misjudge, causing missed coverage.

Problems with Manual Decision‑Making

Traditional manual risk assessment follows three steps: (1) review delivery data and reports, (2) make a decision to proceed or request additional QA, and (3) monitor post‑release bugs and conduct case studies. This process suffers from high data‑gathering cost, inconsistent expertise, and risk of knowledge loss when personnel leave.

Proposed Risk‑Driven Delivery Framework

The solution replaces manual judgment with a machine‑learning‑driven pipeline composed of three core components:

Risk Identification : collect data from five dimensions (over 50 features) such as test tickets, requirement IDs, pipeline IDs, and custom business features; establish lineage to feed the model.

Risk Control : automatically recommend test activities, select or generate test cases, and construct test inputs based on identified risks.

Risk Decision : combine rule‑based logic, a statistical model, and impact assessment to output risk probability, severity, and actionable recommendations.

Risk Identification Details

Data collection spans code changes, personnel, project scope, and impact range. The system ingests these features into a model that learns risk patterns from historical test outcomes.

Risk Control Mechanism

Based on identified risks, the system suggests targeted test activities, automatically generates missing test cases, and executes them to maximize coverage while minimizing redundant effort.

Risk Decision Modeling

The authors treat risk prediction as a binary classification problem and select logistic regression for its interpretability, low computational cost, and suitability for limited training data. The model formula is p = 1 / (1 + exp(-θ·x)), where θ are learned weights and x are feature values such as development duration or changed function count.

Model performance is evaluated with accuracy, precision, and recall. A risk matrix (probability vs. impact) guides actions: high‑probability/high‑impact risks are intercepted, low‑risk items flow automatically, and intermediate cases require QA confirmation.

Visualization and Feedback Loop

A risk‑visualization report presents risk data, decision conclusions, and suggested actions. QA can provide feedback, which is fed back into the model for continuous improvement.

Deployment Results

Q3 2022: identified 1,123 non‑autonomous test projects and intercepted 318 bugs.

Saved 2,172 person‑days by automating 4,345 autonomous test projects; waiting time for autonomous test evaluation dropped from 50 h to 2 h.

Future Direction

The current system operates at an “assisted decision” level (analogous to L2 in autonomous driving). The roadmap aims to progress toward conditional and fully autonomous decision‑making, ultimately achieving end‑to‑end automated delivery.