Code Coverage Best Practices

This article originates from the Google Engineering Blog, published on August 17, 2020, titled “Code Coverage Best Practices,” authored by Carlos Arguelles, Marko Ivanković, and Adam Bender.

Key take‑aways:

Code coverage brings significant benefits to developers' workflows.

Efforts to increase coverage usually foster an excellent engineering culture.

Pursuing a high coverage percentage alone does not guarantee high‑quality testing.

Coverage data highlights how much code remains uncovered.

There is no universal "ideal" coverage number that applies to all products.

Teams should strive to improve coverage comprehensively.

Do not obsess over moving coverage from 90% to 95%.

Human judgment about the risk of uncovered code is more important than raw percentages.

Even low‑coverage products can take concrete, incremental steps to improve.

Targets like 90% overall are often unreasonable, but aiming for 99% on each commit is sensible.

Integration and system test coverage is also important.

Set gates to prevent code that does not meet coverage standards from reaching production.

The full text explains that after decades of improving software testing at large companies, the authors advocate using coverage data to assess risk and find testing gaps. They acknowledge the controversy around coverage value and the polarized debates it sparks.

They provide tools and discussion points to help teams find common ground and use coverage information effectively. The best‑practice recommendations include:

Coverage adds measurable, objective value to the development workflow. While not a perfect quality metric, it offers a standard that requires little manual effort and is supported by tools for most languages. However, it is an indirect, lossy indicator and should be combined with other metrics.

Increasing coverage often cultivates an excellent engineering culture. Teams that prioritize coverage treat testing as a first‑class citizen, design for testability, write more modular code, and improve overall health and operational excellence.

Chasing high percentages alone is misleading. Focusing on reaching 100% can create a false sense of security, waste resources, and lead to low‑value tests. Coverage does not guarantee that covered lines are correctly tested; mutation testing is suggested as a better technique.

Coverage data should highlight what is *not* covered. Even with low overall coverage, the risk of uncovered code reaching production is higher and should be addressed.

No single "ideal" coverage number fits all products. Desired coverage depends on business impact, change frequency, code lifespan, complexity, and domain. Investment in infrastructure to simplify testing is required to reach higher targets.

Strive for overall improvement. Google’s guidance suggests 60% as acceptable, 75% commendable, and 90% exemplary, but teams should set values that match their needs.

Avoid obsessing over marginal gains. Gains diminish logarithmically after a point; focus on moving from low to moderate coverage (e.g., 30% to 70%) and ensure new code meets thresholds.

Human judgment of uncovered code matters more than line‑percentage. Discussing specific uncovered lines during code review is more valuable than chasing arbitrary numbers.

Low‑coverage products can still improve incrementally. Apply the "scout rule" – keep the codebase clean and improve coverage step by step.

Ensure frequently changed code stays covered. While 90% overall may be unrealistic, aiming for 99% on each commit is reasonable, with 90% as a lower bound.

Integration and system test coverage matters. Combining unit, integration, and end‑to‑end coverage gives a fuller picture and prevents false confidence.

Set coverage gates. Teams should define gating mechanisms (overall vs. new code, static thresholds, incremental gates) and enforce them without turning coverage into a checkbox that creates perverse incentives.

In summary, code coverage is a useful, objective metric that can drive better engineering practices when used alongside other quality indicators, with a focus on risk reduction, incremental improvement, and thoughtful gating.

(End of article)