Beyond Feature Delivery: Evaluating Team Contributions in AI-Augmented Development

In teams using AI agents, a developer who ships several features is highly visible, while another who spends days fixing test environments, adding project constraints, and cleaning up flaky checks may have no demoable output at review time. Counting only requirements, code, and commits disadvantages the latter; swapping to rule counts, tool numbers, or model calls merely changes the counting method.

Tools Shipped: First Verify Actual Adoption

Harness here refers to the environment and mechanisms that support agent task execution—project context, tools, permissions, validation, and recovery. Improving these capabilities can prevent the same errors across multiple projects. For example, if import-function test data lacks isolation causing cross-test interference, fixing initialization and cleanup mechanisms may save several teams debugging time. Such work deserves recognition, but benefits must be verified with the adopting teams.

However, claiming “ten teams use it, so benefits multiply by ten” is flawed. Some teams never faced the issue, some haven’t onboarded, some only downloaded a template. Actual adoption, effect, and maintenance cost must be recorded separately. Concrete questions: which repositories integrated, does the original failure still occur, has debugging time decreased, how much extra maintenance does the new mechanism add? Missing data remains a pending judgment.

Clarifying Contributions Beats a Single Score

Microsoft Research’s SPACE framework evaluates developer productivity across Satisfaction, Performance, Activity, Communication, and Efficiency, rejecting a single activity number. This mindset can be borrowed, but each team must define evaluation criteria based on its own role divisions. For agent-using teams, five discussable contribution categories emerge:

Business Delivery

Evidence: Acceptance results, quality, post-delivery actual usage.

Pitfall: More features ≠ greater value.

Shared Improvements

Evidence: Adoption scope, before/after similar incidents, reuse feedback.

Pitfall: Publishing a tool ≠ anyone uses it.

Engineering Judgment

Evidence: Risk identification, design trade-offs, subsequent validation.

Pitfall: Don’t inflate contribution with hypothetical accident losses.

Collaboration & Mentoring

Evidence: Handoff smoothness, whether others can continue independently.

Pitfall: Don’t count shares and meetings as proxy.

Long-Term Maintenance

Evidence: Upgrades, incident response, deprecation and handover status.

Pitfall: More firefighting ≠ better.

This table need not be scored for everyone. People responsible for business delivery, platform maintenance, or junior members gaining experience inherently carry different workloads. Assess based on assigned tasks and resources first, then on execution quality.

Team Improvements Cannot Directly Become Individual Rankings

A shorter project cycle may stem from clearer requirements, more stable tests, and fewer approvals simultaneously. Attributing the change entirely to an agent tool or a single person distorts reality. DORA metrics focus on application/service performance; using team-level metrics to rank individuals obscures who drove the improvement and encourages cherry-picking easy tasks while hiding bad news.

Work opportunities differ: core-repo maintainers touch many teams, while frontline developers handle hard-to-standardize details. The former’s longer user list doesn’t automatically mean larger contribution. Peer and user feedback can supplement, but must be specific: what problem was solved, what friction exists in usage, can you continue without them? Avoid turning it into a popularity vote.

Fewer Rules Can Signal Better Work

Models and tools evolve; guidance once necessary may become obsolete. Duplicate rules increase context load; outdated automated checks can block normal work. Therefore, deleting a verified-unnecessary rule, merging two conflicting processes, or disabling a plugin with no benefit are valuable maintenance acts. Rewarding only new additions while leaving cleanup unrewarded ensures no one does it.

Deletion requires evidence: confirm what problem the rule originally prevented, then compare before/after behavior in a controlled scope, preserving necessary permissions and quality constraints. Don’t disable all protections just because the model got stronger. This work often lacks feature screenshots but reduces long-term team burden.

Don’t Let Platform Maintenance Rely on One Enthusiast

Someone willing to share tools is a good start. But if original demands don’t decrease and everyone’s environment issues land on that person, sharing turns into unpaid overtime. Teams must define maintenance scope, response arrangements, and allocated time. Personal reference scripts and critical dependencies cannot carry the same responsibility. Key capabilities need a designated successor, upgrade and deprecation plans.

Similarly, if onboarding is a team need, it should enter planning and role definitions—not be penalized at review time because “personal delivery dropped.” A dedicated platform department isn’t mandatory; small teams can achieve this through rotation and clear ownership.

During reviews, besides asking “what features shipped this month,” also ask “which problems no longer require repeated handling?” Evidence-backed improvements deserve recognition, and ongoing maintenance must be scheduled into work hours. This is more practical than inventing an “AI Engineer Composite Score.”

The discussion so far centers on development and business processes. In industrial settings, beyond maintenance, a harder constraint appears: when the network goes down, which work can still continue?

References

Microsoft Research, The SPACE of Developer Productivity: There’s more to it than you think : https://www.microsoft.com/en-us/research/publication/the-space-of-developer-productivity-theres-more-to-it-than-you-think/

DORA, DORA’s software delivery performance metrics : https://dora.dev/guides/dora-metrics/