Boosting Mobile App Quality: ZhiZhuan’s Scalable Code Coverage Solution

Background

Code coverage is a key metric in software testing that measures how much source code is exercised by tests. It helps teams identify untested code, reduce potential bugs, and improve code quality. In mobile app development, high iteration costs and difficult post‑release fixes make coverage especially important for safe releases.

ZhiZhuan Group previously built a custom client‑side coverage solution but faced challenges such as slow reporting on Android due to full‑instrumentation and incompatibility with Kotlin, as well as iOS limitations with Objective‑C and Swift. Frequent production incidents prompted an upgrade to a high‑availability solution, which is described in detail below.

Practice and Exploration

Existing online articles on coverage are often outdated or lack concrete code. ZhiZhuan needed a solution supporting all languages and merging coverage data for full visibility, so they redesigned the pipeline from integration to collection, reporting, merging, and visualization.

2.1 Android Code Coverage

2.1.1 Solution Principle

The original Android solution used a self‑developed instrumentation approach inspired by event‑tracking: insert probes during compilation, record execution of logical blocks, report data at appropriate times, and aggregate results on the backend using tag dimensions.

Instrumentation: insert probe code at compile time.

Probe: each probe records whether a logical block was executed.

Data reporting: probes upload local coverage records to the server.

Data aggregation: backend merges reports across devices, branches, and builds using tags.

2.1.2 Instrumentation Implementation

During compilation, probes generate a globally unique ID and a mapping file that is uploaded to the backend. At runtime, executed probes mark the corresponding IDs as covered and report them.

Mapping file format

<code>ClassName
  MethodName#MethodPathMD5#MethodSignature
  GlobalUniqueBlockID#LocalBlockNumber#LineNumber</code>

Mapping file example

Instrumentation flow diagram

2.1.3 Solution Upgrade

The early approach applied full‑instrumentation to all code, causing performance overhead and large backend computation time. The upgraded approach instruments only the diff between the current branch and master , reducing mapping size and upload volume, which dramatically speeds up backend aggregation and enables near‑real‑time coverage views.

Full‑instrumentation scheme

Incremental instrumentation scheme

Key incremental workflow

Change file acquisition : obtain diff between current branch and master via backend API, parse Java and Kotlin file paths.

Incremental instrumentation : instrument only classes whose source files appear in the diff.

Mapping file upload : send incremental mapping files to the backend.

Coverage reporting : at runtime, report coverage indices for changed code.

Merge calculation : backend merges reports from multiple branches, builds, and runs.

2.2 iOS Code Coverage

2.2.1 Solution Selection

Earlier iOS coverage relied on gcov, which suffered from performance, stability, and Swift incompatibility. After evaluating tools, llvm‑cov (based on clang/llvm) was chosen for its multi‑language support and powerful customization.

Implementation

Overview

Advantages

Disadvantages

Xcode built‑in coverage

Enable "Gather coverage data"

1. No extra tools needed

2. Friendly UI

1. Cannot customize format

2. Cannot export to server

gcov

GCC‑based

None

1. No Swift support

2. Low efficiency

llvm‑cov

Clang/LLVM‑based

1. Supports C, C++, Obj‑C, Swift, etc.

2. Strong customizability

Complex to use

2.2.2 Build Machine Configuration

Coverage can be toggled via a CI/CD flag (e.g., COVERAGE_SWITCH=true ). All test builds except the production package enable coverage by default.

2.2.3 Pod Configuration

The project uses CocoaPods; llvm‑cov instrumentation is added through pod scripts. Both main project and component code are instrumented via pod configuration.

pre_install

During pre_install , the diff API is called to obtain the list of files that need incremental instrumentation.

post_install

Separate configurations are applied for the main project and components to distinguish coverage data.

<code># Set pre‑compile variable USE_COVERAGE_SERVICE
config.build_settings['GCC_PREPROCESSOR_DEFINITIONS'] = '$(inherited) USE_COVERAGE_SERVICE'
# Swift coverage flags
config.build_settings['OTHER_SWIFT_FLAGS'] = '$(inherited) -profile-generate -profile-coverage-mapping'
# Link profiling library
config.build_settings['OTHER_LDFLAGS'] = '$(inherited) -fprofile-instr-generate'
# Disable optimizations for correct mapping
config.build_settings['SWIFT_OPTIMIZATION_LEVEL'] = '-Onone'
config.build_settings['GCC_OPTIMIZATION_LEVEL'] = '0'
config.build_settings['ENABLE_DEBUG_DYLIB'] = 'NO'
</code>

For Objective‑C files, the COMPILER_FLAGS are adjusted to include profiling flags only when needed:

<code>if diff_file_array.include?(file_name)
  file.settings ||= {}
  current_flags = file.settings['COMPILER_FLAGS'] || ''
  flags_array = current_flags.is_a?(Array) ? current_flags : current_flags.split(/\s+/)
  # Add profiling flags if missing
  flags_array << '-fprofile-instr-generate' unless flags_array.include?('-fprofile-instr-generate')
  flags_array << '-fcoverage-mapping' unless flags_array.include?('-fcoverage-mapping')
end
</code>

Swift does not support file‑level instrumentation; duplicate reports are deduplicated on the server.

2.2.4 Collection & Reporting

Spawn a background thread with a timer to collect profraw data.

Use __llvm_profile_write_file to write and compress the data.

Upload the compressed file with branch metadata for server‑side merging.

Add duplicate‑report detection and fault‑tolerance handling.

Integration and Usage Experience Optimization

Many applications in ZhiZhuan gradually adopt the coverage plugin, but integration involves plugin setup, script configuration, and language compatibility. After enabling incremental coverage, numerous optimizations were applied.

3.1 Integration Optimization

3.1.1 Configuration Simplification

Problem: Manual configuration errors (project name, group name) caused plugin failures.

Solution: Automate configuration script to fetch project settings, reducing manual steps.

3.1.2 Enhanced Build Logs

Problem: Long build processes with sparse logs made troubleshooting difficult.

Solution: Add detailed logs for configuration parameters, diff API calls, and mapping writes.

3.2 Usage Optimization

3.2.1 Language Compatibility

Problem: Kotlin bytecode (lambdas, by‑lazy, coroutines, high‑order functions) and Swift had instrumentation gaps.

Solution: Implement Kotlin bytecode instrumentation adapters for lambdas and by‑lazy blocks.

3.2.2 Coverage Detail Display

Problem: Coverage view lacked context, showing only changed lines.

Solution: Front‑end now provides expandable context code for changed sections.

3.2.3 Reporting Exception Handling

Problem: Executed code sometimes did not appear in coverage due to instrumentation, mapping, network, or backend merge issues.

Solution: On device, show an error banner with retry and detail view for failed uploads.

3.3 Troubleshooting Tools

3.3.1 Debug Plugin

A browser plugin visualizes backend data and mapping files, improving debugging efficiency.

3.3.2 Uninstrumented Code Highlight

Problem: Uninstrumented executable code appeared as "no coverage" without explanation.

Solution: Front‑end now highlights such code, allowing users to quickly identify instrumentation gaps.

Online Results

1. Each build in the engineering management platform generates coverage data, enabling branch‑level merging.

2. Coverage dashboards show overall, main‑project, and component coverage, with drill‑down capability.

3. Detailed line‑level coverage view uses color coding for quick inspection.

Conclusion and Outlook

The coverage plugin is now stable across multiple core ZhiZhuan applications, providing clear visibility into test coverage and supporting safe releases.

Future work includes extending coverage to unlinked component projects, adding support for HarmonyOS, and further improving integration experience.

Beyond coverage, ZhiZhuan is advancing AI‑assisted code review in CI/CD pipelines and planning automated anomaly analysis to continuously strengthen code quality and delivery capability.