Why Frontend Coverage Reports Were Wrong and How NYC Fixed Them

Background

Code coverage measures the proportion of source code executed by tests, providing a metric for test completeness. Unlike static analysis, coverage focuses on dynamic execution and is crucial for quality control, especially when self‑tested features exceed 80% of requirements.

Prerequisite Knowledge

Basic Coverage Metrics

Line Coverage : Percentage of source lines executed at least once.

Function Coverage : Percentage of functions executed at least once.

Branch Coverage : Percentage of conditional branches executed.

Path Coverage : Percentage of possible execution paths covered.

Code Instrumentation

Instrumentation inserts probes into the code to record execution data. It can be performed at compile time (e.g., via a Babel plugin in a Webpack build) or at runtime (e.g., in Node.js).

Before instrumentation:

function istanbul_test() {
    console.log('istanbul_test');
}
function istanbul_test2() {
    console.log('istanbul_test2');
}
var test = false;
if (test) {
    istanbul_test();
} else {
    istanbul_test2();
}

After instrumentation (simplified):

var __cov_... = (Function('return this'))();
if (!__cov_....__coverage__) { __cov_....__coverage__ = {}; }
__cov_... = __cov_....__coverage__;
// ...instrumented counters for statements, functions, branches...

The instrumented code adds a global coverage object ( __cov_...) with statementMap, fnMap, and branchMap to track execution counts.

Initial Frontend Coverage Scheme

The original scheme collected only line coverage for incremental changes. It used a custom Webpack plugin that injected babel-plugin-istanbul during the Beetle build process, then reported coverage data from window.__coverage__ to a backend service.

Problems Identified

Incorrect source line numbers in reports.

Coverage data sometimes showed uncovered lines that were actually executed.

Three bad cases were observed: vue2 + decorators , vue2.6 + composition API , and vue2.7 + setup API .

Analysis revealed that the issue stemmed from misinterpreting Istanbul’s statementMap and s arrays. The AST‑based instrumentation does not map one‑to‑one with editor line numbers, especially in .vue files that combine template, script, and style sections.

Solution Approach

Local Debugging Workflow

Add scripts to package.json:

"istanbulDev": "cross-env ISTANBUL=1 npm run dev",
"reportCoverage": "nyc report --reporter=html"

Configure nyc in package.json to include js, ts, and vue files and exclude node_modules and tests.

"nyc": {
  "include": ["src/**/*.{js,ts,vue}"],
  "excludeAfterRemap": false,
  "exclude": ["node_modules", "tests"],
  "extension": [".js", ".vue", ".ts"],
  "report-dir": "./coverage",
  "temp-directory": "./.nyc_output"
}

Create diffFileList.conf to list incremental files.

Install nyc as a dev dependency.

Run pnpm run istanbulDev, interact with the pages listed in diffFileList.conf, and capture window.__coverage__ in the console.

Save the captured JSON into a timestamped file under .nyc_output.

Execute pnpm run reportCoverage to generate an HTML report.

Final Scheme

The refined pipeline uses NYC to generate clover‑format reports, which accurately reflect line coverage for .js, .ts, and .vue files. The process includes:

Pre‑processing source code and change data after deployment.

Collecting raw coverage data from the browser and merging it asynchronously on the server.

Optimizing upload frequency to avoid redundant data.

Generating incremental coverage by merging s execution counts per commit.

Data Pre‑processing

After deployment, the service downloads the repository, computes code diffs, and stores change metadata in a database to feed NYC later.

Coverage Data Collection and Merging

Clients report full coverage payloads every 10 seconds. The backend periodically merges these payloads per commit, summing execution counts in the s array. For repeated uploads, the system now deduplicates data to reduce load.

statementMap maps statement IDs to source line/column ranges; s records execution counts for each statement.

Generating Coverage Results

When a coverage query is made, the service copies merged data into .nyc_output and runs:

npx nyc --include "src/**/*.{js,ts,vue}" --exclude-after-remap false --exclude "node_modules" --exclude "tests" --extension .js --extension .vue --extension .ts --report-dir ./coverage --temp-directory ./.nyc_output report --reporter=clover

The clover XML format provides per‑file, per‑line execution counts, which are then combined with the previously stored diff information to compute incremental line coverage.

Lines generated by rendering code that do not appear in the clover file are marked as covered.

Files containing only non‑instrumentable template or CSS code are also marked as covered.

Conclusion

By switching to NYC for report generation, the team resolved the mismatched line‑number issue caused by misinterpreting Istanbul’s raw data. Accurate coverage metrics now enable better test planning, higher code quality, and more reliable releases.