How to Scientifically Measure Software Team Performance: Key Metrics Explained

As technology managers we often have to answer questions such as why a cost centre needs many people, why a request takes a long time to go live, why our team does not follow the 996 work schedule, and how we demonstrate our value and efficiency.

These questions can feel frustrating. The debate over "should we adopt 996?" is essentially a derivative of the "man‑month myth": adding people or hours to finish software faster. In the short term, if goals are clear and business needs demand speed, extra effort may help, but treating it as a panacea will exhaust the team and drive talent away.

Therefore we discuss how to measure team performance scientifically. Note that the discussion focuses on team‑level performance; individual performance in the HuoCheBang tech team is handled with KPI and OKR.

First, Measurement

Only what can be measured can be assessed and improved.

Quantifying the output of an internet‑scale technology team is difficult. Unlike manufacturing, software work is intangible and recorded as intangible assets on the balance sheet.

Moreover, software development differs from construction: two qualified teams given the same design will not produce identical code, and the effort required to implement a feature is highly subjective, especially in agile environments where testing, development, and planning overlap.

The core problem is the inherent complexity of software, which makes workload estimation hard.

Problems with Traditional Methods

Historically, performance measurement focused on productivity, which has several issues:

It emphasizes total output rather than useful output.

It looks at individual output instead of team output.

It confuses productivity with true performance.

Below are some common traditional metrics.

Lines of Code (LOC)

Many Chinese companies, including Huawei, use LOC as an indicator. While fewer lines for the same problem is generally better, overly terse code can become unreadable and unmaintainable, creating technical debt.

Story Points

Story points measure a team’s velocity but do not reflect productivity. Using them for performance has two drawbacks:

They are relative values only comparable within the same team and context.

When tied to performance, teams may inflate completed story points at the expense of useful work and collaboration.

Recorded Work Hours

Tracking work hours shows resource utilization, but high utilization often means the team has no time to respond to changes, perform optimizations, or address technical debt. Queueing theory tells us that when utilization reaches 100%, lead time tends toward infinity.

How We Should Measure

A good measurement approach should focus on two key aspects:

Overall effective output of the whole team (e.g., avoiding hand‑offs that degrade overall value).

Useful work rather than total work.

From this perspective we identified four primary metrics:

Delivery Lead Time

Deployment Frequency

Mean Time To Restore Service (MTTR)

Change Fail Rate

Lead Time

In Lean Theory, lead time is the interval from a user’s request to its fulfillment. In software it consists of two parts: the time to confirm the requirement and design a solution, and the time from implementation to testing and production release. The latter is easier to measure, so we track the time from code commit to production deployment.

Deployment Frequency

Because software batch size is hard to define, we look at how often deployments occur. Desired frequencies range from multiple times per hour to daily or weekly, depending on the product and delivery model.

MTTR (Mean Time To Restore)

Beyond delivery speed, we need to gauge system stability. Traditional MTBF is less useful for complex modern systems; instead we measure how quickly the service can be restored after a failure.

Change Fail Rate

This metric tracks the proportion of changes (releases or configuration updates) that cause service degradation, require hot‑fixes, rollbacks, or patches.

Additional Supporting Metrics

After defining the four core metrics, we also monitor quantifiable factors that influence them:

Average Work‑In‑Progress (WIP) Tasks

Number of Jira tasks currently in the "In Progress" state. Keeping WIP below 2.0 helps avoid overload and bottlenecks.

Code Scale

Commit count in the repository over the past three months. More code is not inherently better; excessive code creates maintenance overhead.

Average Feature Count

Number of configuration files or features per project per engineer. A high average indicates heavy maintenance load, potentially reducing quality.

Bug Count

Total bugs per project. More bugs increase requirement‑to‑completion time and iteration cycles.

Expected Time to Complete All Squeezed Demands

Calculated as average requirement‑to‑completion time multiplied by the number of squeezed demands. Large backlogs increase lead time, reduce planning flexibility, and lower team morale.

After establishing these metrics, we integrated Jira, Confluence, and ReviewBoard to collect data automatically and non‑intrusively, creating a comprehensive dashboard without disrupting daily work.

Author: @lenciel