How Precise Testing Transforms Quality Assurance at NetEase Yanxuan

What is Precise Testing

Precise testing, first proposed in 2016, aims to make the testing process visual, intelligent, trustworthy and accurate, turning testing from experience‑driven to data‑driven.

Goals

The core idea is to use precise and intelligent software to replace manual, experience‑based testing, reducing test cases to targeted ones, improving efficiency and lowering missed‑defect risk.

Core: Bidirectional Traceability

Forward traceability: Developers can view the code details of test cases to facilitate defect fixing.

Reverse traceability: Testers can quickly determine the scope of test cases from changed source code, reducing blind regression work.

Precise Testing in Yanxuan

Yanxuan integrates precise testing throughout the requirement lifecycle, focusing on “what to test” and “how well it is tested”, supporting both manual and automated scenarios.

Submission Phase

After a version is submitted, code analysis identifies interface‑level impact, guiding test planning and automatically matching relevant automated test cases.

Testing Phase

Coverage analysis after test execution shows the test completeness of changed code, helping testers fill gaps and deepen code understanding.

Regression Phase

The in‑house quality platform runs regression suites and provides incremental and full‑coverage reports.

Release Phase

CI triggers coverage gate checks; if thresholds are not met, the release is blocked.

Platform Architecture

The platform builds on open‑source tools such as JaCoCo, JGit, ASM, and Maven, adds a wrapper layer, and uses a factory‑pattern design to compose independent components. APIs expose functionality to the quality platform and other callers.

Code Processing

Pull project code for branch switching, compilation, diff calculation, ASM scanning and code coloring.

Diff calculation using JGit and JavaParser to identify changed classes and methods.

Compile code with Maven/Gradle for subsequent ASM scanning.

Static Code Analysis

Bytecode scanning with ASM (preferred for performance) or Javassist/Bcel.

Generate global method call graph, filtering out irrelevant methods.

Recursively resolve call relationships, bridge interfaces to implementations, and handle anonymous inner classes.

Filter out noise using whitelist annotations.

Dependency Analysis

The service‑dependency platform reveals upstream and downstream services affected by changed interfaces.

Intelligent Test‑Case Recommendation

For each changed unit, suitable automated test cases are selected from the test‑case library and presented on a visual panel.

Breakthroughs

JaCoCo Coverage for Incremental Changes

We extended JaCoCo to generate coverage reports for only the changed code, handling exec file merging and incremental reporting.

Persisting Coverage Across Deployments

By modifying the JaCoCo agent to dump coverage data on shutdown and merge with historical data, we retain coverage information across service restarts.

Dynamic Analysis for Polymorphism and AOP

Static scanning cannot capture polymorphic or AOP calls; we use a Java‑agent to record real method traces during automated or functional tests, supplementing static analysis.

Future Directions

Usability: Improve integration speed, compatibility, performance, visualization, mobile support, and expose core capabilities as services.

Precision: Enhance code coloring, real‑traffic dynamic analysis, and more accurate dependency impact assessment.

Intelligence: Combine precise analysis, automation, and coverage to create a closed loop that reduces test cases, recommends optimal ones, and raises test quality.