Why Using Unit Test Coverage as a Hard Gate Fails and What Actually Works
The article critiques the practice of enforcing unit‑test coverage as a strict quality gate, explains why both overall and incremental coverage thresholds can be gamed, and proposes showing incremental coverage in merge requests while improving overall coverage through broader engineering processes.
Some improvements arise organically from the bottom up, while others are imposed from the top down; using unit‑test coverage as a blunt quality gate is an example of the latter.
In a push for company‑wide agile, DevOps, and development efficiency, teams often try to shift testing left: developers are expected to write good unit tests, raise coverage, and embed the coverage metric into a quality gate that blocks any workflow that does not meet the threshold – a seemingly perfect solution.
However, this logic chain contains several flaws, especially in the final step of making coverage a hard gate.
When a merge request (or pull request) includes a coverage requirement, the request is blocked if the overall coverage does not reach a predefined threshold. If the whole‑project coverage is already close to the threshold, a lazy developer can skip writing tests for a new feature, causing only a tiny dip (e.g., 0.1%) that still satisfies the gate, rendering it ineffective.
Evaluating only the incremental coverage of the changed code also has problems. For trivial changes that require little logic, writing unit tests may involve more mock code than production code, making the effort disproportionate.
Therefore, neither full‑project nor incremental coverage should be used as an absolute gate. Incremental coverage is still valuable: displaying it in the merge request helps reviewers notice test adequacy without enforcing a strict pass/fail rule.
Overall coverage remains important, but it should be monitored at the project level rather than per merge request. If the global coverage is low, teams can establish mechanisms to ensure new code includes appropriate tests, assign dedicated tasks to address historical debt, and gradually improve the metric.
For a deeper dive, see the book “High‑Quality Efficient Delivery”.
Efficient Ops
This public account is maintained by Xiaotianguo and friends, regularly publishing widely-read original technical articles. We focus on operations transformation and accompany you throughout your operations career, growing together happily.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.