Why AI Speeds Up Some Developers 55% But Slows Others 19%

01 Two Contradictory Experiments

In July 2025, nonprofit research institute METR published a randomized controlled trial with 16 senior open-source developers (average 5 years maintaining million-line repositories) performing 246 real tasks (bug fixes, features, refactors). One group used AI (Cursor Pro with Claude 3.5/3.7 Sonnet), the other did not. The AI group took 19% longer on average. Before the experiment, developers expected AI to make them 24% faster; after being told they were 19% slower, they still felt 20% faster.

Google Cloud's 2024 DORA report surveyed 39,000+ practitioners: 75% felt more efficient with AI, yet every 25% increase in AI adoption correlated with a 1.5% drop in delivery speed and a 7.2% drop in stability .

Conversely, a Microsoft Research and GitHub controlled experiment with 95 professional developers building an HTTP server in JavaScript found the Copilot group finished in 1 hour 11 minutes vs. 2 hours 41 minutes for the control — a 55.8% speedup .

The key difference: the Copilot task had clear boundaries, explicit acceptance criteria, and a standard solution shape . METR's tasks lived in million-line codebases with tangled dependencies and undocumented conventions — constraints that had never been fully written down . Developers accepted only 44% of AI suggestions ; 75% read AI output line by line; 56% heavily rewrote AI-generated code. The time saved typing was eaten by verification, integration, and debugging costs. Notably, slowdown was more pronounced in developers' most familiar repositories because they could spot AI errors and spent time fixing them, whereas in unfamiliar code they missed errors and deferred problems.

02 Two Distinct Problems

The variance stems from two separate issues:

Information problem: AI lacks the tacit constraints living in developers' heads (design rationale, module invariants, historical decisions). Solution: write down context and implicit constraints.

Capability problem (computational irreducibility): Some processes have no shortcuts — you must execute them step by step. Stephen Wolfram demonstrated this with a neural network trained to generate balanced parentheses sequences. Despite high accuracy, the network assigned 15% probability to an invalid closing parenthesis, because counting parentheses requires exact computation, not pattern matching. Wolfram calls this computational irreducibility : no pattern recognition can replace the actual counting process.

AI does pattern recognition, not precise rule execution. Engineering's fatal spots (a single bracket, version number, byte order, timing constraint) demand exactness. Both problems share the same exit: make rules explicit . For the information problem, writing rules gives AI missing context. For the capability problem, writing rules turns them into code and tools that execute precisely, rather than asking AI to guess.

03 AI Eats Tasks, Not Jobs

Stanford Digital Economy Lab's "Canaries in the Coal Mine" (Brynjolfsson, Chandar, Chen) tracked 730+ occupations via ADP payroll data (late 2022 onward). Four layers:

Overall employment hasn't collapsed.

Employment for 22–25 year olds in high-AI-exposure occupations is ~19% below expected; 41–49 year olds' employment rises. In software, entry-level hiring down ~20%, senior demand up ~20%.

Mechanism is "not hiring" not "firing" — junior roles simply aren't posted. Tasks previously done by juniors (information gathering, first-draft code, preliminary research, document assembly) are now routed to a model plus a senior engineer.

Employment decline concentrates in occupations where AI directly completes tasks (generate full code solutions, batch standardized replies, template documents). Occupations requiring creativity, complex judgment, multi-party trade-offs see stable or growing employment.

The study defines "standardized" operationally: clear boundaries, explicit acceptance criteria, no ad-hoc judgment — i.e., the portion that can be written down . Researchers hypothesize AI excels at codified knowledge (writable, teachable, repeatable) but struggles with tacit knowledge (experience-derived, inarticulable).

04 What "Rules" Actually Means

Rules = turning judgment into machine-checkable artifacts. Three layers, from farthest to nearest to your judgment:

Tool rules: Connect external precise executors (e.g., Wolfram|Alpha for math, CI checks for conventions, schema validation for APIs, pipeline steps for config changes). Turns personal memory into team asset.

Model rules: Model selection, parameters, context length, temperature. Same model with different invocation rules can yield order-of-magnitude quality differences.

Constraint rules: Output structure, boundary conditions, when to stop and ask a human. Most like writing SOPs for a new hire; often omitted, leading to hallucinated outputs.

Constraint rules are most valuable because they encode your unique understanding of "what correct looks like." Counterintuitively, rules are not restrictions — they are both an AI capability amplifier and a solidifier of your own capability . Writing a rule converts a one-off manual act into an infinitely repeatable, tireless execution.

05 Two Ways to Create Rules

Posture 1: You define, AI executes. Default path. Your skill ceiling becomes the output ceiling because AI cannot know rules you haven't written.

Posture 2: Co-create rules with AI. Severely underestimated. AI's systematic errors reveal your implicit rules. Example: translating a long document, the author noticed recurring terminology inconsistencies and figure-text mismatches only after AI made patterned mistakes. Tracing those errors backward surfaced the unwritten style guide, which was then codified into a glossary and validation rules. Systematic errors are how implicit rules expose themselves. This works for any task done three-plus times with consistent steps. It's not delegating responsibility to AI; it's using AI as a mirror.

06 What Part of Expertise Is Replicable?

Replicable: The portion you've already written down — methodologies, checklists, judgment criteria, "if-then" rules. Once in text or code, it can be loaded into a model.

Irreplicable: Knowing which rule to invoke in a given situation. This is the true moat: recognizing exceptions before they happen, foreseeing a reasonable-looking design's failure three months later, detecting cross-department conflicts from a single requirement phrase. These are not knowledge but prioritization skills — ordering "what matters more."

The Stanford data's "senior up, junior down" is not a tenure victory but a "rule-definers vs. rule-executors" advantage . A senior whose only edge is "slightly better/faster execution" faces the same obsolescence curve, just later. A junior who early extracts implicit rules into machine-executable assets accumulates rule assets that compound quickly. WEF Future of Jobs Report 2025: by 2030, 39% of core skills change; 59% of workforce needs reskilling (11% unlikely to get it); 63% of employers cite skills gap as top transformation barrier; 40% expect to reduce headcount due to AI task automation — note "tasks," not "roles," aligning with Stanford.

07 Matthew Effect: Evidence Both Ways

Divergence evidence: Anthropic Economic Index shows AI usage highly correlated with per-capita GDP; gap stable, no catch-up. Top 5 US states account for ~50% of usage with 38% of working-age population. Usage concentrates in computer/math dense areas. Model success rate inversely correlates with task complexity (measured by human expert time). Resource-rich actors already tackle complex tasks — a reinforcing loop.

Convergence evidence: METR's long-term tracking: model reliable task duration doubles every ~7 months (2019–present). Today's expert tasks become routine in two years. METR couldn't run a second trial because they couldn't recruit senior developers willing to work without AI even at $150/hr — tool-as-default arrives faster than imagined. Anthropic projects US state usage gaps could close in 2–5 years, an order of magnitude faster than 20th-century tech diffusion.

Together: divergence is happening, but the window hasn't closed and moves fast. "First out" isn't the resource-poor — it's the immobile. Tool access equalizes rapidly; the scarce skill is knowing where to place rules .

08 Concrete Actions

Identify the "writable" parts of your work. Brutal heuristic: anything you've done 3+ times with essentially identical steps is a candidate (weekly report format, fixed review questions, incident troubleshooting sequence, requirement decomposition method). Ask honestly: "Am I still doing this manually?" If yes, you have a rule waiting to be extracted.

Write it down. Start as prose, evolve to checklist, then to code or tool call. Each step is a leverage multiplier. This is the true meaning of "AI as code": not AI writing code, but you writing your work understanding into machine-executable form. AI is executor and amplifier; you remain the definer — and that definer's premium rises.

Build new rules on top of automated ones. When a task is automated, attention frees up. Invest it in more same-level tasks → you're just a more efficient executor (accelerating self-consumption). Invest in "previously impossible because too costly" tasks → you become a rule creator (compound interest).

09 On "Asking Questions"

Einstein: "Formulating a problem is often more essential than its solution." Previously, the gap between question and answer required months of human labor, so solvers were valued. Now, question-to-draft-answer cost is unprecedentedly low, re-pricing question-asking upward. But a bare "can we do better?" creates no value. High-value questions carry constraints: boundaries, success criteria, verification methods. A high-quality question is itself a rule draft. When you clarify what you want, don't want, and how success is measured, the rule is half-written; AI fills the rest.

Conclusion

METR's slowed developers weren't incompetent — they faced tasks that could not be fully articulated , so every AI snippet required their personal judgment to verify, and that judgment was exactly what AI lacked. Copilot's 55.8% faster developers faced a fully specifiable task , so rules were ready-made and AI consumed them directly. The difference was never the tool; it was whether the task had been clearly articulated. And articulation can only be done by humans.

The era's easiest mistake: treating "irreplaceability" as comfort. Noticing AI's current gaps and feeling safe because "AI needs human judgment" is useless — gaps shrink fast. Today it miscounts parentheses; in three years it may not. The useful mindset isn't "what can't AI do?" but:

What do I have that can be written as code and handed to a machine for infinite execution?

That answer is your asset. Then ask one level deeper:

After shipping those, what new rules will I define next?

That answer is your direction.

Data Sources

METR, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity , July 2025

Google Cloud, 2024 DORA Report (39,000+ practitioners)

Peng et al., The Impact of AI on Developer Productivity: Evidence from GitHub Copilot (arXiv:2302.06590), 2023

Cui et al., The Productivity Effects of Generative AI: Evidence from a Field Experiment with GitHub Copilot , MIT × Microsoft × Accenture

METR, Measuring AI Ability to Complete Long Tasks (arXiv:2503.14499), March 2025

Brynjolfsson, Chandar & Chen, Canaries in the Coal Mine? Six Facts about the Recent Employment Effects of Artificial Intelligence , Stanford Digital Economy Lab

World Economic Forum, Future of Jobs Report 2025

Anthropic, Economic Index series

Stephen Wolfram, What Is ChatGPT Doing … and Why Does It Work? , February 2023