How Shentong Achieved Near‑Complete Test Coverage: Practices, Tools, and Lessons

Introduction

“Is the testing finished?” is a common question asked to system testing staff, reflecting confidence in test results. Complete testing is impossible due to constraints, but testing teams strive for near‑complete coverage.

Preparation Work

Shentong’s technical quality team introduced a “Quality Keeper” system that standardizes test design, execution, and regression, adding workflow automation and traceability.

Manual Test Effectiveness

Given the ROI of manual testing, the team focuses on functional test completeness, improving test case design by retaining essential elements (steps, expected results) and using tools to streamline case creation.

Test Case Design

Traditional test cases contain eight elements: ID, module, title, priority, preconditions, inputs, steps, expected results. Shentong refined this to a concise template that emphasizes steps and outcomes while supporting normal, abnormal, and non‑functional cases.

Code Coverage

The team built a JaCoCo‑based coverage plugin (SCC) to collect and persist coverage data across deployments, addressing data loss and class‑conflict reset issues by persisting raw data and performing method‑level clearing.

Key Issues

Data persistence and class‑conflict clearing were solved by storing raw coverage and slicing data at the method level, preventing whole‑class coverage reset.

Regression and Precise Testing

To avoid costly full regression, Shentong combines static analysis with change impact identification, linking changed code to affected APIs and test cases, enabling targeted regression.

Automation Test Effectiveness

With thousands of automated cases, the team evaluates effectiveness through interface, code, and scenario coverage, prioritizing services based on monitoring data and generating realistic scenario tests from production inputs.

Optimization Based on Issues

Feedback from the upgraded technical service team feeds back into the testing process, forming a PDCA loop that improves coverage, regression, and overall quality.

Future Plans

Future work includes front‑end coverage identification, impact analysis for data and configuration changes, and an AI testing assistant to assist analysis and automation.