Why High Test Coverage Can Mislead You: Lessons from Java Code

Introduction

Automated testing has become a cornerstone of quality‑focused development, and Java developers now have a rich ecosystem of tools—static analysis, code‑metric dashboards, and coverage reporters—to assess code health. However, raw coverage numbers can be deceptive if you do not understand what they actually measure.

Understanding Coverage Types

Most tools report line (statement) coverage , which counts the percentage of executable lines that were exercised during a test run. Branch coverage goes a step further by measuring whether each decision point (e.g., if or logical &&/||) was taken in both true and false directions.

High percentages only indicate that many lines were executed; they do not guarantee that the exercised code behaved correctly or that edge cases were examined.

Illustrative Java Example

The following Hierarchy class models a simple class‑hierarchy structure and is used throughout the article to show how coverage tools work.

package com.vanward.adana.hierarchy;

import java.util.ArrayList;
import java.util.Collection;
import java.util.Iterator;

public class Hierarchy {
    private Collection classes;
    private Class baseClass;

    public Hierarchy() {
        super();
        this.classes = new ArrayList();
    }

    public void addClass(final Class clzz) {
        this.classes.add(clzz);
    }

    /**
     * @return an array of class names as Strings
     */
    public String[] getHierarchyClassNames() {
        final String[] names = new String[this.classes.size()];
        int x = 0;
        for (Iterator iter = this.classes.iterator(); iter.hasNext();) {
            Class clzz = (Class) iter.next();
            names[x++] = clzz.getName();
        }
        return names;
    }

    public Class getBaseClass() { return baseClass; }
    public void setBaseClass(final Class baseClass) { this.baseClass = baseClass; }
}

A companion HierarchyBuilder creates a Hierarchy instance by walking up the superclass chain.

package com.vanward.adana.hierarchy;

public class HierarchyBuilder {
    private HierarchyBuilder() { super(); }

    public static Hierarchy buildHierarchy(final String clzzName) throws ClassNotFoundException {
        final Class clzz = Class.forName(clzzName, false, HierarchyBuilder.class.getClassLoader());
        return buildHierarchy(clzz);
    }

    public static Hierarchy buildHierarchy(Class clzz) {
        if (clzz == null) {
            throw new RuntimeException("Class parameter can not be null");
        }
        final Hierarchy hier = new Hierarchy();
        hier.setBaseClass(clzz);
        while ((clzz.getSuperclass() != null) && (!clzz.getSuperclass().getName().equals("java.lang.Object"))) {
            clzz = clzz.getSuperclass();
            hier.addClass(clzz);
        }
        return hier;
    }
}

JUnit Test Suite

Initial tests exercised only a subset of the builder’s logic, leading Cobertura to report 59 % line coverage and 75 % branch coverage.

package test.com.vanward.adana.hierarchy;

import com.vanward.adana.hierarchy.Hierarchy;
import com.vanward.adana.hierarchy.HierarchyBuilder;
import junit.framework.TestCase;

public class HierarchyBuilderTest extends TestCase {
    public void testBuildHierarchyValueNotNull() {
        Hierarchy hier = HierarchyBuilder.buildHierarchy(HierarchyBuilderTest.class);
        assertNotNull("object was null", hier);
    }
    public void testBuildHierarchyName() {
        Hierarchy hier = HierarchyBuilder.buildHierarchy(HierarchyBuilderTest.class);
        assertEquals("should be junit.framework.Assert", "junit.framework.Assert", hier.getHierarchyClassNames()[1]);
    }
    public void testBuildHierarchyNameAgain() {
        Hierarchy hier = HierarchyBuilder.buildHierarchy(HierarchyBuilderTest.class);
        assertEquals("should be junit.framework.TestCase", "junit.framework.TestCase", hier.getHierarchyClassNames()[0]);
    }
}

Improving Coverage

Adding more test cases—especially ones that invoke the buildHierarchy(String) overload and verify the private constructor’s behavior—raised the reported coverage to 88 % line coverage, but still left the private constructor untested.

public void testBuildHierarchyStrName() throws Exception {
    Hierarchy hier = HierarchyBuilder.buildHierarchy("test.com.vanward.adana.hierarchy.HierarchyBuilderTest");
    assertEquals("should be junit.framework.Assert", "junit.framework.Assert", hier.getHierarchyClassNames()[1]);
}

public void testBuildHierarchyWithNull() {
    try {
        HierarchyBuilder.buildHierarchy((Class) null);
        fail("RuntimeException not thrown");
    } catch (RuntimeException e) {
        // expected
    }
}

Common Pitfalls Demonstrated

A deliberately buggy method PathCoverage.pathExample(boolean) returns null when the condition is false, causing a NullPointerException after trim(). The coverage tool shows 100 % line coverage because the single branch is exercised, yet the defect remains hidden.

public String pathExample(boolean condition) {
    String value = null;
    if (condition) {
        value = " " + condition + " ";
    }
    return value.trim(); // NPE if condition is false
}

Best Practices for Using Coverage Data

Estimate effort for modifying existing code by correlating uncovered lines with change‑impact analysis.

Assess overall code quality; high coverage reduces defect risk but does not eliminate it.

Guide functional‑test planning by spotting areas with low or missing coverage.

Treat coverage reports as a diagnostic tool: look for low‑coverage hotspots, investigate why certain branches are never exercised, and add targeted tests rather than chasing a high percentage for its own sake.

Conclusion

Test‑coverage metrics are valuable when used to expose untested code paths, but they must be interpreted alongside thoughtful test design. By combining precise coverage data with disciplined test authoring, teams can improve confidence, reduce regression risk, and make more accurate estimates for future development work.