Fine-Grained Code Coverage Integration and Practices at iQIYI

Code test coverage is a metric that describes how much of the source code is exercised by tests. It is a white‑box testing technique that can reveal uncovered code blocks. iQIYI’s development and QA teams have explored ways to integrate and use coverage data to improve code quality.

Most coverage metrics are reported at an overall level, lacking granularity. If coverage can be measured at the requirement‑level and linked to owners, testing efficiency and precision can be greatly improved.

This article describes how iQIYI combines code management, project management, and DevOps tools to achieve fine‑grained coverage statistics.

Background

Agile and iterative development models are common in Internet companies, enabling rapid market feedback. However, as projects iterate, codebases become large and complex, making code quality critical. Coverage data is a useful indicator of code quality, and integrating it into the development cycle is worth exploring.

Definition

Code coverage (Code coverage) measures the proportion of source code executed by tests.

Significance of Code Coverage

1. Improves test cases: Coverage reports show which code has not been executed, prompting analysis of missing test scenarios.

2. Enhances code quality: Developers can see execution paths, understand logic, and identify dead code.

3. Provides a quantitative quality standard: Coverage gives a concrete number to assess code quality.

Current Issues

Inability to distinguish incremental code.

High usage cost.

Difficulty linking coverage to specific requirements or owners.

High maintenance cost due to frequent changes in collaborative test environments.

Goals and Features

iQIYI built on an existing coverage generation tool, integrating DevOps utilities to quickly generate coverage reports with low cost, supporting branch‑level and requirement‑level incremental coverage.

Business Architecture

Features:

Simplified integration: Users only need to push generated Java, class, and exec files to a designated location to obtain detailed reports automatically.

Incremental coverage: Diff algorithms separate incremental code and generate separate incremental and full reports.

Branch‑level coverage: Calculates coverage for a specific code branch.

Requirement‑level coverage: Calculates coverage for a Jira issue by linking branch naming conventions to requirements.

[Incremental & Full Report]

[Branch‑Level Report]

[Requirement‑Level Report]

Technical Implementation

Using Java as an example, iQIYI chose the open‑source JaCoCo tool for coverage generation.

[Running Principle]

JaCoCo requires three inputs: source Java files, compiled class files, and execution data (exec) generated by probe instrumentation.

Source and class files are pushed via CI jobs; exec files are exported from the JVM via TCP.

The three artifacts are aggregated to produce a full coverage report.

GitLab diff between test and master branches identifies incremental code, which is combined with the other artifacts to generate an incremental report.

Commit history links incremental code to its originating branch, enabling branch‑level reports.

Jira issues are linked to branches to produce requirement‑level coverage.

Core Enhancements

JaCoCo adaptation: Added scheduled reporting over TCP socket client to avoid data loss on JVM crashes.

Incremental coverage generation: Used Git diff and ASM to compute coverage for added/modified code.

Branch‑level coverage: Traced commits to associate coverage with specific branches.

Requirement‑level coverage: Enforced branch naming conventions to map branches to Jira issues.

Coverage Standards

Type

Standard

Version delivery

All incremental coverage ≥ 90%

Test pass

Requirement branch incremental coverage ≥ 90%

Code release

Branch incremental coverage ≥ 90%

High coverage does not guarantee high quality, but low coverage indicates untested code that may cause stability issues. Over‑pursuing coverage can lead to unnecessary effort.

Requirement Testing and Release Process

[Requirement Testing & Release Flow]

Testers create test cases based on requirements, execute them, and evaluate the generated coverage report. If coverage does not meet the threshold, additional test cases or code fixes are required before approval and production deployment.

Summary & Outlook

The article presents iQIYI’s exploration of code coverage, including tool adaptation, DevOps integration, and multi‑dimensional reporting (incremental, branch, requirement). After deployment, backend service coverage rose from below 70% to over 90%, improving service reliability. Future work includes generating coverage from production traffic, using it to drive automated test case selection, and comparing production vs. test environment coverage.