How We Built an Incremental JaCoCo Coverage Tool for Faster DevOps Feedback

Background

In a large micro‑service environment with hundreds of Java applications, overall unit‑test and integration‑test coverage exceeds 70 %. However, new projects often have low functional‑test coverage and developers cannot rely on total coverage to assess the completeness of testing for newly added code. An incremental code‑coverage tool was built to provide a metric that focuses on the changed code.

Solution Design

The tool is based on JaCoCo, an open‑source JVM coverage library, and runs in on‑the‑fly mode using the -javaagent option. The workflow consists of four steps:

Collect the JaCoCo .exec file generated after test execution.

Compute the diff between a baseline commit and the target commit.

Parse the diff and split it to method‑level granularity.

Generate a coverage report that includes only the changed methods.

JaCoCo Modification

JaCoCo’s instrumentation logic uses ASM. To keep the change minimal, only the visitMethod method of ClassProbesAdapter was overridden so that it processes probes for methods identified as added or modified, while all other classes and methods are ignored.

public void visitMethod(int access, String name, String descriptor, String signature, String[] exceptions) {
    // custom logic to filter methods based on diff analysis
}

Obtaining the Exec File

During QA deployment the Java application is started with a JaCoCo agent configured as:

-javaagent:jacocoagent.jar=output=tcpserver,address=0.0.0.0,port=XXXX

This streams execution data over TCP, allowing the tool to retrieve the binary exec data without stopping the JVM. The following method demonstrates how the exec data is dumped via a socket connection:

public void dumpData(String localRepoDir, List<IcovRequest> icovRequestList) throws IOException {
    // validate requests
    icovRequestList.parallelStream().forEach(req -> {
        // open socket to the address/port defined in the request
        // use ExecutionDataWriter and RemoteControlReader/Writer to retrieve exec data
    });
}

Extracting Diff Code to Method Granularity

JGit is used to clone the repository, compute the diff, filter out non‑Java files, and map changed source files to compiled class files. For each changed file the tool determines added, deleted, or modified methods via a custom MethodDiff utility.

private List<AnalyzeRequest> findDiffClasses(IcovRequest request) throws GitAPIException, IOException {
    String gitAppName = DiffService.extractAppNameFrom(request.getRepoURL());
    String gitDir = workDirFor(localRepoDir, request) + File.separator + gitAppName;
    DiffService.cloneBranch(request.getRepoURL(), gitDir, request.getBranch());
    String baseCommit = DiffService.getCommitId(gitDir);
    List<DiffEntry> diffs = DiffService.diffList(request.getRepoURL(), gitDir, request.getNowCommit(), baseCommit);
    List<AnalyzeRequest> diffClasses = new ArrayList<>();
    for (DiffEntry diff : diffs) {
        if (diff.getChangeType() == DiffEntry.ChangeType.DELETE) continue;
        AnalyzeRequest ar = new AnalyzeRequest();
        if (diff.getChangeType() == DiffEntry.ChangeType.ADD) {
            // handle added file
        } else {
            // modified file – compute method level changes
            HashSet<String> changedMethods = MethodDiff.methodDiffInClass(oldPath, newPath);
            ar.setMethodnames(changedMethods);
        }
        String classPath = gitDir + File.separator + diff.getNewPath()
            .replace("src/main/java", "target/classes")
            .replace(".java", ".class");
        ar.setClassesPath(classPath);
        diffClasses.add(ar);
    }
    return diffClasses;
}

Generating the Coverage Report

The modified JaCoCo API parses the exec file together with the list of changed methods and class paths. It builds a coverage model that includes only the incremental code, calculates line‑coverage ratios, and stores the metadata for later retrieval.

private IBundleCoverage analyzeStructure(List<AnalyzeRequest> analyzeRequests, String sourceDirectory) throws IOException {
    CoverageBuilder coverageBuilder = new CoverageBuilder();
    for (AnalyzeRequest ar : analyzeRequests) {
        Analyzer analyzer = new Analyzer(execFileLoader.getExecutionDataStore(), coverageBuilder, ar.getMethodnames());
        File classFile = new File(ar.getClassesPath());
        try (InputStream in = new FileInputStream(classFile)) {
            analyzer.analyzeClass(in, sourceDirectory);
        }
    }
    // aggregate line counters
    long totalCovered = 0, total = 0;
    for (IClassCoverage cc : coverageBuilder.getClasses()) {
        totalCovered += cc.getLineCounter().getCoveredCount();
        total += cc.getLineCounter().getTotalCount();
    }
    double coveredRatio = total == 0 ? 0 : (totalCovered * 100.0 / total);
    // persist coverage data (e.g., via DAO) – omitted for brevity
    return coverageBuilder.getBundle("incremental");
}

Effect

The generated report displays coverage statistics only for the methods that were added or modified. Unchanged methods are shown with 0 % coverage, allowing developers to focus on testing gaps introduced by the latest changes.

Integration with DevOps

The incremental coverage tool is integrated into the internal DevOps platform. After functional testing in the QA environment, the platform triggers the report generation through asynchronous batch APIs. The resulting report URL is stored in the DevOps system, enabling developers to view incremental coverage directly from the platform.