Precise Testing Platform: Practice, Data, and Insights
A precise testing platform built on AST analysis and a recommendation engine was deployed across four core applications, enabling automated test execution with 100% pass rates, at least 75% service coverage, around 80% precision, early defect interception, and up to one hour per‑iteration time savings while planning further expansion and precision improvements.
This article introduces a precise testing platform used in a multi‑domain development environment to improve test coverage and reliability.
Background: As the number of jointly built domains grows, code complexity increases, making it difficult to accurately assess the impact of code changes. The platform helps QA identify the interface scope of each version and raises testing efficiency.
Practice Overview: Starting from a pilot in the second quarter, the platform was expanded to four core applications in the third quarter. Iterative data and experiences are presented.
Testing Process: Before testing, the team confirms the changed services, verifies interface changes with developers, and checks platform precision. During testing, automated cases are executed to achieve 100% pass rate and at least 75% service coverage.
Positive Outcomes: The platform enriched test cases, uncovered hidden issues, and improved efficiency. Example 1 shows a missed interface that was detected and fixed, preventing a potential production problem.
Metrics: Precision rate is calculated as a/(b+c)*100% (a = recommended changed interfaces, b = new interfaces, c = modified old interfaces). Recent iterations show precision around 80% (e.g., version 528: 15+ changes, 15+ recommendations, ~80% precision).
Data Highlights: Versions 527, 526, 525, and 524 reported interface change counts, left‑shifted test cases, recommendation numbers, and precision percentages ranging from 63% to 81%.
Platform Architecture: The platform builds method call chains using Abstract Syntax Tree (AST) analysis, extracts changed and impacted interfaces, and feeds them into a recommendation engine that matches automated, functional, and loss‑prevention test cases. It also integrates coverage metrics from code‑coverage, automation, and test‑case platforms.
Experience Summary: Over the third quarter, the team achieved 100% recommendation coverage for many versions, intercepted defects early, and saved 0.5‑1 hour per iteration per person through automated planning and execution.
Future Outlook: Ongoing optimization aims to raise precision further, expand the platform to more applications, and continue improving test efficiency and maintenance costs.
DeWu Technology
A platform for sharing and discussing tech knowledge, guiding you toward the cloud of technology.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.