Incremental Code Coverage with Diff Engine and JaCoCo for Precise Testing

A good programmer considers all logical branches, but exhaustive manual testing often misses cases, leading to heavy regression testing burdens. Companies attempt various automation solutions, yet many fail to provide convincing results or meaningful coverage feedback.

We envision a visual coverage feedback system that guides test strategy without relying on subjective QA experience, allowing precise testing of only impacted code and providing objective coverage metrics for both manual and automated tests.

The solution comprises several modules: a Diff engine for multi‑dimensional code change analysis, an automated development platform, a JaCoCo‑based coverage component supporting incremental and full coverage monitoring, and Jenkins to orchestrate the workflow.

Workflow steps:

Extract code change information from Git and convert it to JSON using Python.

Transform line‑level changes to method‑level changes via AST (Abstract Syntax Tree) analysis.

Identify changed methods and notify relevant personnel for manual and automated testing.

Generate coverage data using JaCoCo, which inserts probe booleans into bytecode via ASM without altering program behavior.

Integrate JaCoCo Maven plugin with Jenkins so that coverage reports are produced automatically.

Method‑level diff is defined by marking a method as changed if any line within its range is modified. AST provides each method’s line range, parameters, and annotations, enabling precise identification of changed interfaces and methods.

Full‑coverage reports often lack relevance for a specific change, showing low coverage despite thorough testing. The root cause is that they aggregate coverage across the entire project, not just the modified logic.

We evaluated two approaches: post‑processing the HTML report (rejected due to complexity and incomplete data) and pre‑processing the coverage data by excluding unchanged code before analysis (chosen for lower cost and higher accuracy).

Implementation details include JaCoCo’s probe insertion (offline vs. on‑the‑fly), handling of multiple test runs via Dump, Merge, and Report commands, and careful method overload comparison.

Integrating the Diff engine with Jenkins allows automatic generation of incremental coverage reports, hiding unchanged packages/classes and highlighting increased coverage for modified components, which aids both testers and project managers in assessing test progress and release risk.

Challenges such as empty reports on master‑branch regressions were solved by adding version‑level diff, ensuring incremental coverage is produced even when code does not change between branches.

Overall, the approach combines precise diff‑driven testing with continuous integration, reducing manual effort, improving coverage credibility, and enabling a seamless handoff between automated and manual testing phases.