How iQIYI Built a Precise Testing System to Boost Regression Quality and Efficiency

Why Precise Testing?

Regression testing often suffers from high cost and low accuracy because even small feature changes can affect a large set of historic functionalities. Traditional approaches rely on manual scope selection, which is subjective and risky; iQIYI data shows that 50% of defects stem from historical feature interactions.

Core Idea of Precise Testing

Precise testing records the mapping between test cases and the code they execute using a code‑coverage tool, stores this information in a knowledge base, and when code changes occur, queries the knowledge base to recommend only the affected test cases, thereby improving efficiency while reducing quality risk.

System Characteristics

Multi‑scenario support: works for both client and server, manual and automated testing, and integrates with multiple internal platforms.

Implementation level: captures code coverage at the test‑case level, supports system‑wide linking on the server side and multi‑component aggregation on the client side.

Precision: links test cases to code at branch or code‑block granularity.

Implementation Steps

1. Collecting Test‑Case to Code Mapping

The custom code‑coverage tool is installed automatically during the build process for Android, iOS, and server components. It instruments bytecode to tag each request with a unique identifier, captures coverage data per request, and sends build artifacts and coverage results to the precise‑testing service.

After test execution, the service parses coverage data to determine which classes, methods, and statements each test exercised, storing this mapping in the knowledge base.

Key challenges and solutions:

Server‑side single‑case collection: Parallel testing in shared environments caused coverage mixing. iQIYI first tried isolated environments (high cost) and then developed a self‑built coverage tool that supports concurrent users by tagging each request.

Micro‑service linking: To achieve system‑level coverage, a TraceID is injected into request headers, allowing the service to correlate calls across micro‑services and link them to the originating test case.

Client multi‑component aggregation: Componentized mobile apps require aggregating coverage from the main app and each SDK/POD. The build platform uploads each component’s artifacts; the service parses dependency files (e.g., podfile) to locate and merge component coverage.

Test‑Range Recommendation

When a change is detected, the system extracts the start and end Git commits, identifies modified files, and queries the knowledge base to compute the precise set of impacted test cases. Recommendations can be triggered manually or automatically via CI/CD pipelines.

Two enhancements address recommendation noise:

Noise reduction: Define “base” test cases for common steps (e.g., opening the home page). Only base cases are recommended when shared steps change, reducing redundant recommendations.

Branch‑level precision: Instead of method‑level mapping, the system records branch conditions and only recommends tests whose executed branch actually changed, eliminating unrelated test cases.

Results and Impact

After deployment, the precise testing system achieved significant regression test reduction without compromising coverage. In manual testing scenarios across multiple business lines, the number of executed cases dropped noticeably while no regression gaps were observed. For automated testing, the system filtered out high‑frequency request paths, applied an additional precision layer, and reduced both CI execution time and result‑analysis effort.

Future Plans

iQIYI intends to extend precise testing to more scenarios, such as gray‑release monitoring, front‑back integration testing with full‑stack coverage, intelligent recommendation (e.g., automatic equivalence class generation and priority ranking), and tighter bug‑to‑code‑traceability to aid defect localization.