How to Measure Software Quality: Essential Metrics and Calculation Methods

1 Overview

This document provides measurement criteria for software quality, test quality, and delivery quality, aimed at product, R&D, testing, project managers, and developers. By setting targets, tracking processes, and analyzing results, it offers data to evaluate current product quality and to build a foundation for future testing and process improvement.

1.1 Background

The guidelines serve as a reference for assessing software testing quality, process quality, and delivery quality.

1.2 Intended Audience

Testers, engineers, project managers, product managers, and developers.

1.3 Reading Guide

Software testing quality metrics focus on data measurement for tested products in R&D and support projects. Test process quality metrics provide data for project managers, product managers, and test leads to evaluate test execution quality. Delivery quality metrics assess the quality of new requirement deliveries.

2 Software Quality Metrics

2.1 Requirement Function Point Coverage Rate

Definition: Ratio of total test cases to the total number of corresponding function points, indicating whether any function points were missed.

Formula: Σ(test case count) / Σ(function points)

Data Sources: Product Requirement Document (PRD), Requirement Traceability Matrix.

Example Result: Coverage = 113 / 8 = 14.13%

2.2 Test Case Execution Rate

Definition: Ratio of executed test cases to total test cases, showing any missed or ineffective executions.

Formula: Σ(executed test cases) / Σ(total test cases) × 100%

Data Source: Test Progress Tracking Sheet.

Example Result: Execution Rate = 100%.

Table summary: All modules (XX模块) reported 100% execution rate, with a total of 113 test cases executed out of 113.

2.3 Defect Fix Rate

Definition: Ratio of closed defects to total effective defects.

Formula: Σ(closed defects) / Σ(effective defects)

Data Source: CPaaS.

Example Result: Fix Rate = 206 / 216 × 100% = 95%.

2.4 Defect Residual Count

Definition: Number of defects pending assignment, resolution, or reprocessing.

Formula: Pending assignment + pending resolution + reprocessing defects.

Data Source: Jira.

Example Result: Residual defects = 10, with no P1 or P2 bugs.

2.5 Regression Pass Rate

Definition: Ratio of defects passed in regression to total defects pending regression.

Formula: Σ(passed regression defects) / Σ(total regression defects) × 100%

Data Source: Internal defect management system (Jira).

Example Result: Pass Rate = 57 / 72 × 100% = 79.2%.

2.6 Module Defect Rate

Definition: Ratio of defects in each module to the total number of defects, used to assess module quality.

Formula: Σ(module defects) / Σ(total defects) × 100%.

Data Source: Jira.

Defect distribution visualized in the following images:

2.7 Severe Defect Rate

Definition: Ratio of severe defects in each module to total defects.

Formula: Σ(severe defects in module) / Σ(total severe defects) × 100%.

Data Source: Jira.

Severity distribution illustrated below:

2.8 Defect Density and Convergence

Definition: Ratio of defects per tested module for each version, indicating version stability.

Formula: Σ(defects in version) / Σ(test modules in version).

Data Source: Jira.

Key charts:

3 Test Process Quality Metrics

3.1 Internal Defect Discovery Rate

Definition: Ratio of defects found internally to the sum of internal and user‑found defects, reflecting internal detection capability.

Formula: Σ(internal defects) / Σ(internal + user defects) × 100%.

Data Source: Jira.

Result: 80 / (80 + 5) = 94%.

3.2 Defect Effectiveness Rate

Definition: Ratio of bugs confirmed by developers to total reported bugs, indicating testing quality.

Formula: Σ(effective defects) / Σ(total defects) × 100%.

Data Source: Jira.

Example table (summarized):

Tester Zhang – 60 effective / 62 total = 97%; Tester Li – 40 / 42 = 95%; Overall – 100 / 104 = 96%.

3.3 Test Case Execution Efficiency

Definition: Ratio of executed test cases to execution time (hours), measuring tester efficiency.

Formula: Σ(executed cases) / Σ(execution hours).

Data Source: Jira.

Example: Zhang – 30 cases / 12 h = 2.5; Li – 20 / 7 h = 2.8; Overall – 50 / 19 h = 2.6.

3.4 Defect Discovery Efficiency

Definition: Ratio of defects discovered to total testing time, indicating defects found per hour.

Formula: Σ(submitted defects) / Σ(effective testing hours).

Data Source: Jira.

Example: Zhang – 25 / 30 h = 0.83; Li – 10 / 9 h = 1.1; Overall – 35 / 39 h = 0.9.

4 Delivery Quality Metrics

4.1 Release Success Rate

Definition: Ratio of successfully released versions to planned releases.

Formula: Successful releases / Planned releases × 100%.

Data Source: Release reports.

Result: (15‑1) / 15 × 100% = 93%.

4.2 Fault Rollback Rate

Definition: Ratio of versions that required rollback due to faults to planned releases.

Formula: Rollback versions / Planned releases × 100%.

Data Source: Release and monitoring reports.

Result: 2 / 16 × 100% = 12%.