AI Coding Accelerates: How R&D Delivery Systems Must Evolve

Bottleneck Moved

Software delivery has never been just about "writing code". A change from idea to production goes through requirements, design, implementation, verification, release, and runtime observation. Each step produces new facts that may overturn previous judgments. Code is merely the most visible segment.

DORA's 2025 research calls AI an amplifier of organizational capability. Teams with clear boundaries, reliable tests, and complete platform capabilities more easily turn generation speed into delivery speed; teams with vague requirements, fragile systems, and manual release approvals often just get more code awaiting review.

METR ran a controlled experiment on mature open-source projects. Under specific sample conditions and early-2025 tooling, developers familiar with the repository took 19% longer to complete tasks when using AI. This does not mean AI slows development, but shows that faster local actions do not equal earlier task completion.

Architecturally, this resembles fitting a larger pump into a pipeline. Flow accelerates, and previously unnoticed narrow sections, bends, and valves quickly become new constraints.

After AI widens the coding segment, the bottlenecks of the R&D system shift both upstream and downstream: upstream must express intent more precisely, downstream must produce credible evidence faster.

Thus, candidate code multiplies while the delivery queue may lengthen. Generation works at machine speed, but verification, decision, and release still queue in the old way, turning new capacity into pending changes.

The R&D system ultimately delivers at the pace of its slowest segment, not its fastest.

Figure 1: Local acceleration pushes queues downstream; end-to-end speedup requires state, version, and evidence to move forward together.

Bottlenecks Hide in Handoffs

Consider a common requirement: add bulk export to an order list.

A development Agent can quickly add the export API, async task table, and frontend button. A test Agent can generate a batch of test cases. Each artifact appears complete in isolation.

Problems emerge after code is handed off: which orders are exportable, who has permission, how long files are retained; whether the new async API breaks old clients that depend on synchronous error codes; how to verify large files and timeouts in a test environment with only a few hundred rows; whether the running service has loaded the new config after release.

These issues span requirements, code, environment, and runtime systems — they belong to no single Agent.

If each stage passes only a document or a "done" message, information thins across handoffs. Downstream Agents must re-read the repo, guess context, and add checks. Humans fall back to scrolling chat logs, hunting meeting notes, and confirming which version was actually validated.

Chaining more Agents does not fix this breakage. The key is not the number of roles, but whether a change's goal, state, and evidence can move forward together.

The end-to-end focus is not about one Agent owning all nine phases. The chain connects when, at each boundary, the upstream step leaves facts clear enough that the downstream step need not re-guess.

Turn Delivery into a State Machine

Traditional R&D processes are organized by role: product writes requirements, architects design, developers code, testers verify, ops releases. This division suits team organization but not Agent system design.

An Agent does not need a more complex org chart. It needs to know which change it is processing, what state that change is in, and what is missing to reach the next state.

When we previously dissected Scion, we split a source-code change into "persisted, merged, verified, deployable". Extending that to the full delivery chain adds steps forward and backward:

Current Intent → Candidate Change → Persisted → Merged → Verified → Deployable → Effective

Inventing more state names adds no value. The point is to avoid a single "done" meaning seven different things.

Code saved does not mean it entered the shared mainline; mainline merged does not mean tests covered the current snapshot; build deployed does not mean the running instance loaded the new version. Every state transition must answer three questions: which immutable version changed, what is the evidence, and where does it stop on failure.

To make this chain run, three types of information must stay together: state tells where it is, evidence tells why this step holds, and risk gates decide who can push it forward.

Each advancement records old state, new state, object version, evidence reference, and initiator. Later troubleshooting follows the state log to see which step approved which version and on what basis.

At this point, the core object of the R&D system is no longer an Agent's conversation but a continuously advancing change. Agents can be swapped, tasks interrupted, execution environments destroyed. State and evidence persist, so the next run picks up where it left off.

Specs Follow Facts

Agents still need specs before starting. Goals, constraints, and acceptance criteria must be explicit, or the model guesses.

Specs are not written once at kickoff. Requirements express current intent, code and dependencies encode existing constraints, and the runtime environment continuously yields new facts. When they conflict, the system must update its understanding, not ignore reality to follow the original plan.

Addy Osmani, discussing Agent specs, emphasizes goals, boundaries, tests, and project structure. Karpathy's Context Engineering is not just prompt writing but packing task description, examples, retrieval results, tools, state, and history into a limited context window.

In an R&D system, a spec resembles an intent version that updates with facts while leaving a trail of every change. Goals and acceptance criteria are relatively stable; design and task breakdown evolve with evidence. If an Agent discovers an old interface still called by external clients, compatibility constraints are added back; if an existing async task framework can be reused, the change scope shrinks accordingly.

The value of a spec lies not in its length but in every change having a traceable reason. Later readers can see which decisions came from business goals, which adjustments from code facts, which limits from the runtime environment.

Stuffing the entire requirements, all repository knowledge, and full history into context at once is usually unstable. At each step, only the relevant slice is selected: designing the bulk export API needs permissions, data scope, and compatibility constraints; reaching release needs build version, config changes, and rollback conditions. Context follows state, so critical constraints are less likely to be missed or drowned in irrelevant material.

Harness Retains the Scene

Long tasks rarely finish in one go. Agents read repos, call tools, modify files, wait for tests, and encounter permission errors, dependency failures, context compression, and task handovers.

Keeping only the final reply loses the true execution scene.

Harness preserves these ephemeral states: which step the task reached, what tools were called, what changes were produced, what results are awaited, where safe retries are possible, and where external systems must be re-read first.

This part may not look like "intelligence", but it determines whether the system can recover. After a test timeout, the state might be failure or still running; a release request with no response might have already created a new version remotely. Collapsing all these into "failed, retry" easily creates duplicate resources or double releases.

Execution state and business facts must be separated.

Harness saves the execution scene; the business system saves the authoritative facts. Harness can remember "release request sent, awaiting read-back", but must not unilaterally write business state as "release failed" just because a request timed out.

Models excel at ambiguous problems; state machines excel at guarding deterministic boundaries. Separating them lets Agents persist in uncertain R&D tasks without corrupting external state.

Evidence Binds Versions

The more code is generated, the less verification can rely on the generator's self-attestation.

"Unit tests passed" must state which commit they cover; "deployment succeeded" must state which build runs in the target environment; "config updated" must confirm the instance actually loaded it. Evidence without bound version, environment, and timestamp becomes useless shortly after.

Addy Osmani's "Agentic Code Quality" notes that as Agents generate more code, quality checks shift into Harness, environment, and deterministic constraints. AgentSWE's published failure trajectories are typical: implementations that only fit visible examples, runtime appearances that are complete but internally hollow, and claims of completion without evidence.

The earlier verification moves to deterministic tools outside the model, the more reliable downstream conclusions become. Interface contracts go to tests; build artifacts record hashes and provenance; external actions are verified by reading back resource IDs; post-release, real traffic and metrics are checked. Only when results require semantic judgment should a model evaluate, preserving the materials and criteria it referenced.

An evidence chain might look like:

Intent v3 → Code c7 → Build b12 → Test r19 → Release d4 → Runtime read-back o8

The numbers are illustrative; the reference relationships matter. Test r19 only proves build b12; if code becomes c8, the old test cannot endorse the new version. Release d4 only indicates production effectiveness when the runtime read-back points to the same build.

Completion is not a conclusion from an Agent, but a set of auditable facts.

Figure 2: Intent, code, build, test, release, and runtime read-back point to the same change; high-risk forks divert to humans, new facts feed the next cycle.

Knowledge Also Expires

As Agents enter the full R&D flow, they increasingly depend on project knowledge: architectural constraints, interface conventions, incident records, review comments, historical decisions. These reduce redundant exploration but can also carry stale conclusions into new tasks.

Tencent OK's public practice offers a concrete approach: knowledge carries a TTL; Agents can flag inconsistencies between knowledge and current code; after new commits land on mainline, repository descriptions are incrementally refreshed. This detail addresses the engineering problem more directly than simply counting documented artifacts.

In this system, the knowledge base is closer to a cache; code, configuration, and the runtime system are the sources of truth. A cache without version, source, and update time makes conflicts harder to detect as volume grows. Reliable knowledge entering context should at least be traceable to its origin, the version it targets, and when it needs re-validation. On conflict, the system should not let the Agent arbitrarily pick one version but return to the fact source for reconfirmation.

Such a knowledge loop writes a failure back into specs, tests, or Harness, so the next similar change inherits the new constraint. Organizational capability accumulates here, not in an Agent's ever-growing chat history.

Humans Guard the Forks

Martin Fowler stresses that humans remain responsible for software behavior and implementation. Kief Morris describes the human position shifting from "in the loop" to "on the loop": humans no longer instruct Agents line by line, but design work loops, provide specs, checks, and escalation conditions.

Humans stay in the R&D process, but their attention moves from repetitive execution to the harder-to-delegate forks: why this change, which constraint wins when multiples conflict, which risks can auto-proceed, which actions require human confirmation.

Low-risk, single-repo, easily rolled-back changes can auto-advance within clear boundaries. Changes involving data migration, money, permissions, public APIs, or cross-team commitments pause at a specific state, handing the decision to an authorized person.

This routing need not rely on task name alone. Change scope, rollbackability, data impact, and external commitments can all be gate criteria. The same "modify an interface" differs vastly between an internal experimental endpoint and a public API that customers depend on — they should not share the same automation path.

Humans guard business trade-offs and system boundaries, not act as messengers between every stage.

First Run a Chain Through

End-to-end R&D is easily drawn as a grand diagram: multiple Agents, unified knowledge base, auto-review, auto-release, all at once. The diagram is complete, but the actual state transitions may not have a single working path.

For incremental adoption, I prefer starting with a class of well-bounded requirements, such as internal APIs in a single repo or low-risk config changes. First, get the state, evidence, and escalation conditions flowing for that class, letting Agents read code, make changes, run verification, and stop in a recoverable state on failure.

At this stage, I care more about "how delivery went" than "how much was generated": how long from requirement to verifiable change, how many reworks, how much human wait time, how many unknown states appeared, how long to recover from a failure.

These numbers do not directly answer system design, but they reveal where the queue has moved.

AI coding makes candidate code appear earlier. What truly impacts delivery speed next is whether the R&D system can, along a single change, preserve intent, execution state, verification evidence, and runtime results; and when underlying facts change, know which prior judgments are invalidated.

Judging whether a pipeline is mature may come down to a simple check: after a task interruption, personnel handoff, or deploy timeout, can the next engineer or Agent continue from system facts, or must they still scour chat logs, ask around, and guess the remote state again?

If guessing is still required, the bottleneck is not in writing code.

References

Martin Fowler, Agentic Engineering (https://martinfowler.com/bliki/AgenticEngineering.html)

Kief Morris, Humans and Agents in Software Engineering Loops (https://martinfowler.com/articles/exploring-gen-ai/humans-and-agents.html)

Addy Osmani, How to Write a Good Spec for AI Agents (https://addyo.substack.com/p/how-to-write-a-good-spec-for-ai-agents)

Addy Osmani, Agentic Code Quality (https://addyo.substack.com/p/agentic-code-quality)

Thoughtworks, Preparing Your Team for the Agentic Software Development Life Cycle (https://www.thoughtworks.com/en-au/insights/articles/preparing-your-team-for-agentic-software-development-life-cycle)

Google Cloud DORA, State of AI-assisted Software Development 2025 (https://dora.dev/research/2025/dora-report/)

METR, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (https://metr.org/Early_2025_AI_Experienced_OS_Devs_Study-paper.pdf)

AgentSWE: Can Coding Agents Build the Agent You Actually Want? (https://arxiv.org/abs/2610.13361)

Andrej Karpathy, on Context Engineering (https://x.com/karpathy/status/1937902205765607626)

Peter Steinberger / Rohan Paul, on moving from Prompt to Loop (https://x.com/rohanpaul_ai/status/2064042758109839567)