Canyon: A JavaScript End‑to‑End Test Coverage Solution for Frontend Projects

The article introduces Canyon, an open‑source JavaScript coverage tool built on top of IstanbulJS, designed to fill the gap of end‑to‑end (E2E) test coverage collection for large‑scale front‑end projects at Ctrip.

Background : Traditional IstanbulJS only supports unit‑test coverage. Ctrip’s UI automation platform (Flybirds) and massive simulator clusters require E2E coverage, prompting the development of Canyon to handle high‑concurrency coverage reporting, real‑time aggregation, and detailed reporting.

Introduction : Canyon works via a simple Babel plugin configuration, runs in a Node.js environment, and can be deployed in cloud‑native platforms such as Docker and Kubernetes. It integrates seamlessly with CI/CD tools like GitLab CI and Jenkins.

Key Features include code coverage, instrumentation, test execution and reporting, coverage aggregation, change‑code coverage, and a React Native collection scheme.

Code Coverage : Coverage is achieved by inserting probes into the source code before execution. Example of original code:

// add.js
function add(a, b) {
return a + b
}
module.exports = { add }

After instrumentation the code becomes:

// 这个对象用于计算每个函数和每个语句被执行的次数
const c = (window.__coverage__ = {
f: [0],
s: [0, 0, 0],
});
c.s[0]++
function add(a, b) {
c.f[0]++
c.s[1]++
return a + b
}
c.s[2]++
module.exports = { add }

A simple test verifies the coverage:

// add.cy.js
const { add } = require('./add')
it('adds numbers', () => {
expect(add(2, 3)).to.equal(5)
});

The resulting coverage object after the test is { f: [1], s: [1, 1, 1] } , indicating 100% coverage for this file.

Instrumentation : Canyon uses Babel‑plugin‑Istanbul (or experimental Vite/Swc plugins) to perform compile‑time instrumentation, supporting vanilla JavaScript, Babel, Vite, and Swc projects. Example Babel configuration:

module.exports = {
plugins: [
[
'babel-plugin-istanbul',
{
exclude: ['**/*.spec.js', '**/*.spec.ts', '**/*.spec.tsx', '**/*.spec.jsx'],
},
],
],
};

Successful instrumentation can be verified by searching for __coverage__ in the compiled output.

Testing and Reporting : During Playwright tests, the global window.__coverage__ object is collected and uploaded to the Canyon server. Example Playwright script:

const {chromium} = require('playwright');
const main = async () => {
const browser = await chromium.launch();
const page = await browser.newPage();
await page.goto('http://test.com');
await page.click('button');
await page.fill('input', 'test');
await page.click('text=submit');
const coverage = await page.evaluate(`window.__coverage__`);
upload(coverage);
browser.close();
};
main();

For multi‑page applications (MPA), coverage is reported on the visibilitychange event to avoid loss of data during navigation.

Aggregation : Canyon merges coverage objects from many test runs using a message‑queue based, stateless service suitable for containerized deployment. Example merge function:

function mergeFileCoverage(first, second) {
const ret = JSON.parse(JSON.stringify(first));
delete ret.l;
Object.keys(second.s).forEach(k => { ret.s[k] += second.s[k]; });
Object.keys(second.f).forEach(k => { ret.f[k] += second.f[k]; });
Object.keys(second.b).forEach(k => {
const retArray = ret.b[k];
const secondArray = second.b[k];
for (let i = 0; i < retArray.length; i++) { retArray[i] += secondArray[i]; }
});
return ret;
}

Reporting : Canyon renders coverage reports using a Monaco‑editor based UI, annotating source code with coverage decorations. Example rendering code:

const decorations = useMemo(() => {
if (data) {
const annotateFunctionsList = annotateFunctions(data.coverage, data.sourcecode);
const annotateStatementsList = annotateStatements(data.coverage);
return [...annotateStatementsList, ...annotateFunctionsList].map(i => ({
inlineClassName: 'content-class-found',
startLine: i.startLine,
startCol: i.startCol,
endLine: i.endLine,
endCol: i.endCol,
}));
} else { return []; }
}, [data]);

Change‑Code Coverage : Canyon computes coverage for newly added lines using a compare target and GitLab diff API. Example function to compute line coverage from statement coverage is provided.

function getLineCoverage(statementMap, s) {
const lineMap = Object.create(null);
Object.entries(s).forEach(([st, count]) => {
if (!statementMap[st]) return;
const { line } = statementMap[st].start;
const prevVal = lineMap[line];
if (prevVal === undefined || prevVal < count) { lineMap[line] = count; }
});
return lineMap;
}

React Native Support : The same Babel‑based instrumentation is applied to React Native apps, with coverage data streamed via WebSocket:

// Create WebSocket connection
const socket = new WebSocket('ws://localhost:8080');
socket.onopen = () => { console.log('Connected to coverage WebSocket server'); };
socket.onmessage = event => {
try {
if (JSON.parse(event.data).type === 'getcoverage') { socket.send(JSON.stringify(payload)); }
} catch (e) { console.log(e); }
};
socket.onclose = () => { console.log('Disconnected from coverage WebSocket server'); };

Coverage‑Improvement Priority List : Canyon provides a weighted formula to rank files for coverage improvement, combining production coverage, test coverage gaps, and function count.

Community Promotion : The project is open‑sourced on GitHub, invites contributions, and plans further integration with Playwright, Puppeteer, and Cypress.

Reference links and additional reading are listed at the end of the original article.