Why Metrics Fail and How to Design Effective R&D Efficiency Measurements

Recently the author reflected on a previous piece about how metrics can trap development teams, inspired by an article titled "外卖骑手，困在系统里" and his own earlier work on how R&D efficiency can destroy a team.

Historical metric failures : The "window tax" in 17th‑century England forced homeowners to block windows or create skylights, leading to poor lighting, nearsightedness, and unintended industry growth, illustrating how poorly designed measurements can backfire.

Modern examples : At Hai Di Lao, strict metric‑driven rules (e.g., providing glasses cloths to customers wearing glasses, refilling drinks to a certain level, covering phones with plastic bags) caused employee harassment of customers and degraded user experience.

The author cites Jerry Muller’s book Metrics Trap as further reading on the pitfalls of bad measurement.

Root causes of metric failure include using outdated industrial‑era management mindsets in the digital era, over‑emphasizing easily collected data (e.g., lines of code) while neglecting high‑value but hard‑to‑measure outcomes (e.g., product user value, NPS), and focusing on short‑term, quantifiable targets that crowd out long‑term goals.

Common pitfalls :

Relying on easily obtainable quantitative indicators.

Attempting to measure with a single dimension, ignoring the multi‑faceted nature of software work.

Binding metrics directly to personal KPIs, which encourages gaming the system.

Instead, the author recommends building a multi‑dimensional "radar" of metrics that cross‑validate each other (e.g., combining defect count with delivered story points, or overtime with code impact).

Automation of data collection is essential; manual entry leads to data distortion and undermines analysis. The author illustrates this with the burn‑down chart problem where engineers update task status only near sprint end, producing misleading trends.

He points to Tencent’s "dual‑stream" model, where merging a feature branch automatically triggers a status transition, achieving seamless data flow.

Metrics should be tied to team performance, not individual compensation, to avoid perverse incentives while still providing macro‑level insight for improvement.

Metrics are means, not ends: they must serve clear goals. The example of Sonar adoption shows that measuring "project integration rate" without linking to actual code quality can be hollow; better metrics are average fix time for severe issues or trend of problem growth.

Finally, the author warns against blindly copying large‑company practices or building costly metric data platforms without a defined improvement objective; a focused, insight‑driven approach yields higher ROI.

The article concludes with an invitation to continue the discussion on R&D efficiency metrics.