Precise Mobile Testing Platform: iOS Code Coverage Instrumentation and Incremental Coverage Analysis

1. Background

In recent years, Youzan Retail’s business has grown rapidly. Since 2019 the client app is released weekly, putting pressure on quality assurance. The team therefore researched tools to improve mobile testing quality and efficiency, leading to the development of a precise testing platform for mobile.

Mobile testing mainly includes manual testing and unit testing. Manual testing relies on human understanding and lacks quantitative evaluation, while unit testing requires frequent test‑case updates and has high maintenance cost. The platform aims to combine the advantages of both by providing code‑coverage analysis for manual tests.

This article describes the implementation principles and practice of the platform on iOS.

2. Principles

Code coverage measures the proportion of code executed during testing, covering both unit tests and system tests. Two main metrics are line coverage and branch coverage.

2.1 Code Instrumentation

To collect coverage data, the source code must be instrumented. Objective‑C (a superset of C) can be instrumented using GCC’s instrumentation tools via LLVM. The process generates a .gcno file (instrumentation metadata) during compilation and a .gcda file (execution counts) at runtime.

2.1.1 Basic Block

A basic block (BB) is a sequence of instructions with a single entry and exit. If the first instruction of a BB is executed, all instructions in the block are executed exactly once. Functions consist of multiple BBs.

2.1.2 .gcno File

The .gcno file records:

Checksum information

Absolute source file path

Function names and line numbers

Basic‑block identifiers and line numbers

2.1.3 .gcda File

After the instrumented binary runs, calling __gcov_flush() writes execution statistics to a .gcda file, which contains:

Checksum information

Function names and execution counts

Basic‑block identifiers and execution counts

2.1.4 .info File

LCOV (based on gcov) converts .gcno and .gcda into a human‑readable .info file. The file includes fields such as TN (test name), SF (source file), FN/FNDA (function name and execution count), DA (line number and execution count), LF/LH (total lines / covered lines), and additional incremental fields CA, CF, CH for changed lines.

2.2 iOS Project Configuration

In Xcode’s Build Settings, enable the following flags for the target to turn on instrumentation:

InstrumentProgramFlow = YES

GenerateLegacyTestCoverageFile = YES

During runtime, invoke __gcov_flush() to flush coverage data. The environment variable required for flushing must be set as shown in the original diagram.

Because third‑party libraries are integrated as binaries, they cannot be instrumented directly. The team uses Ruby scripts to modify the shell project and sub‑projects to enable instrumentation for all modules.

3. System Architecture

3.1 Architecture Diagram

The system consists of four layers:

CI Layer : Performs instrumented compilation, builds the app, distributes it, and runs hook scripts. It uploads .gcno files to the data‑collection layer and distributes the app to test devices.

Data Collection Layer : Receives .gcno and .gcda files along with metadata (platform, bundleId, branch, commitId, build, file, uuid, etc.).

Data Parsing Layer : Retrieves the intermediate files, generates full‑coverage and incremental coverage data, and handles line‑number translation.

Data Visualization Layer : Presents coverage results as HTML, XML, or notifications.

3.2 Sequence Diagram

The workflow is:

CI builds the instrumented app and uploads .gcno .

Test devices run the app and upload .gcda .

Users input parameters on the front‑end.

The analyzer fetches the files, generates reports, and notifies stakeholders.

3.3 Coverage Generation

LCOV processes the files to produce .info files. For single‑version coverage, simple LCOV commands are used (derive, filter, merge). For incremental coverage, a custom algorithm based on git diff translates line numbers across versions and marks new/changed lines.

3.3.1 Single‑Version Coverage

Commands (simplified):

lcov -c -d $SOURCE -o $DEST_INFO

lcov -r $SOURCE_INFO '$REGEX' -o $DEST_INFO

lcov -a $SOURCE_INFO_0 -a $SOURCE_INFO_n -o $DEST_INFO

3.3.2 Incremental Coverage

The process includes:

Parsing git diff to obtain added/deleted line ranges.

Parsing the original .info to extract function and line execution data.

Applying line‑number translation (shifting) based on deletions/additions.

Marking newly added lines (CA, CF, CH fields) in the generated .info .

After translation, the aligned .info files can be merged with LCOV, and the extra fields allow reporting of incremental coverage.

3.4 Visualization Output

The visualized reports display:

Overall incremental coverage for the whole codebase.

Per‑file incremental coverage.

Line‑level markers indicating whether a line is new/changed.

This enables developers to quickly locate uncovered changes during regression testing.

4. Usage Scenarios

The platform supports:

Developer Self‑Testing (Incremental) : A git‑hook runs incremental coverage after local testing; the report is attached to the commit for review.

Automated Test Coverage (Full) : After CI runs automated test suites, full coverage is generated to evaluate test suite effectiveness and identify dead code.

Regression Test Coverage (Incremental) : Weekly builds collect coverage from testers; incremental reports highlight uncovered changes before release.

Uncovered code can be classified as:

Not tested – requires targeted test cases.

Exception handling – should be covered by unit tests or simulated in system tests.

Redundant code – should be refactored.

5. Conclusion and Outlook

The precise testing system improves self‑testing quality and test‑case completeness. Future work includes intelligent analysis of coverage reports to provide multi‑dimensional insights.