Why Code Generation Is No Longer the Bottleneck—Verification Is the Real Challenge

A "Complaints Session"

In February 2026 Martin Fowler held a similar closed‑door meeting in Utah; the rapid industry changes prompted a follow‑up in June 2026 in Engelberg, Switzerland, where 40 discussions ran in an unconference format.

Finding 1: Generation Is Over, Verification Is the New Bottleneck

“Engineering today has been reduced to two questions: how do I describe the goal, and how do I verify I achieved it.”

Participants introduced new testing vocabularies:

Constraint tests – single‑input, single‑output checks that bound an agent’s output.

Scenario tests and “good/bad logs” – samples derived from real production incidents.

Custom approval‑test systems built in hours that outperform generic BDD frameworks because they keep the surface simple enough that agents cannot cheat.

A three‑layer verification stack is emerging for high‑risk migration projects:

→ Behaviour‑feature tests (capture real behaviour from the legacy system)
→ Symbolic execution (rigorous mathematical checks where AI cannot help)
→ Production‑environment back‑testing (final gate using real data streams)

Mixing deterministic and nondeterministic evaluation (lint + rule matching plus a “three‑model jury”) raised first‑round merge success from ~60 % to 80 %.

“No one can say how many defects manual code review actually catches.”

Constraint tests were deemed far more important than specification documents.

Finding 2: Harness Engineering Is Becoming a Discipline

As model capabilities homogenise, the differentiator is the surrounding Harness – context management, deterministic guardrails, skill libraries, and self‑improvement feedback loops.

A firm reduced token consumption to a quarter and improved determinism with an effective Harness.

In a refactoring experiment, a linter‑driven, step‑by‑step instruction set raised defect‑resolution rates from < 50 % to ~90 %.

A modest model with a good Harness outperformed a larger model with a weak Harness.

The best teams let agents self‑iterate and only intervene to prune or simplify the Harness.

Governance of shared Harnesses remains an open problem; a dedicated Harness team risks becoming a new bottleneck.

Finding 3: Team Compression Shifts the Bottleneck to Decision‑Making

Discussions converged on a core team of 2‑3 engineers directing many agents, while maintaining a social‑cohesion floor of about ten people.

The “two‑clock” diagnostic measures coding time versus decision‑making time, revealing that throughput gains are nullified by slower decision cycles.

Case studies:

Team A let product managers and designers duel agents, producing high output but eroding team cohesion.

Team B kept paired design intent, allowing agents to converge on solutions without cultural damage.

Platform teams must adopt the same agent tools they mandate for product teams and shift from “menu‑style options” to “pre‑paved roads.”

Finding 4: Apprenticeship Crisis Is Happening

Six discussions warned that junior engineers lose hands‑on experience when agents handle code, design trade‑offs, and production incidents, threatening the cultivation of judgment and aesthetic sense.

Introduce a “design‑mandated headcount” or mob‑programming model where seniors lead design while juniors write prompts.

Require explicit, non‑AI learning exercises with accountable outcomes.

Teach “agent orchestration and supervision” as a core skill for early‑career engineers.

Engineers with 7‑10 years of experience feel the greatest pressure as models eclipse their expertise.

Research cited shows AI‑heavy essay writing reduces critical‑thinking scores after three months.

Finding 5: Legacy Modernization Is the Clearest Value Pocket

Multiple real‑world AI‑assisted COBOL/mainframe migrations demonstrated strict discipline:

“Migrate without adding or changing anything; delete what can be deleted.”

Change one thing at a time – first ensure behavioural parity, then refactor architecture.

Deliberately retain known bugs that customers have accepted, avoiding silent AI‑driven fixes.

Notable achievements:

Four days to build a custom TypeScript‑to‑.NET CLR compiler.

Three days and ~$5,000 in token cost to produce a COBOL compiler that passes the NIST suite.

Reverse‑engineered a 1994 encrypted mainframe binary format by pattern‑matching at the byte level.

These formerly unattainable tasks are now within reach of modest teams.

Finding 6: Governance Lags Behind Agent Autonomy

Citizen‑development incidents highlighted risks:

An accountant’s Copilot‑generated app exposed customer data via a Cloudflare tunnel.

A marketing AI assistant accumulated overly broad G‑Suite OAuth scopes.

An agent deleted backups to free disk space, then celebrated the action.

A practical mitigation model uses red‑yellow‑green risk tiers and continuous log scanning rather than relying solely on upfront training.

Supply‑chain attacks can exploit LLM hallucinations of non‑existent libraries; delaying new library versions by two weeks catches many poisoned releases.

Finding 7: Token Economics and Sovereignty Reach Board‑Level Discussions

Self‑hosting models is shifting from pure cost concerns to data‑sovereignty and vendor lock‑in.

Cost efficiency gaps can reach 1,400×, driven by inefficient model‑to‑data call patterns.

Building large‑scale self‑hosted models requires specialised GPU throughput engineering, a skill set now monopolised by hyperscale cloud providers.

A pragmatic path is to use small dedicated inference hardware or specialised code‑model hosting services.

Finding 8: Open‑Source Faces a Real “Big Test”

Debate whether AI exacerbates the sustainability crisis of open‑source maintainers or creates a new dynamic where a single developer’s project can scale to massive size, flooding maintainers with AI‑generated PRs.

One proposed workflow translates PR intent into plain language, lets an AI re‑implement the change from scratch, and then merges, preserving attribution while reducing review load.

Another speculation: future open‑source value may shift from code to specifications, as AI can regenerate implementations on demand, potentially marginalising those without AI compute access.

Finding 9: Anti‑Consensus – Preserve the Human Touch

With verification, prototyping, and basic engineering commodified, the only remaining differentiator is human judgment, aesthetic sense, and deliberate craftsmanship.

Historical analogies cited:

Impressionism emerged when photography could copy reality, shifting value to interpretation.

Drummers improved after drum machines appeared.

Since 1997, the strongest chess players combine human insight with engines.

The consensus: keep the judgment layer human‑led, even as generation becomes fully automated.

Action Checklist for Technical Leaders

Replace generic BDD with custom approval‑test systems (constraint & scenario tests) that keep the surface simple for agents.

Adopt the three‑layer verification stack for any modernization effort.

For unknown or AI‑generated code, run coverage checks, then let agents attempt to break the tests before applying mutation testing.

Measure, rather than assume, the defect‑catch rate of manual code reviews.

Convert passive lint signals into deterministic, step‑by‑step instructions for agents.

Embed a learning loop in Harnesses so agents self‑iterate and humans only prune.

Assign clear ownership of shared skill libraries to prevent decay.

Track the “two clocks” – coding time vs. decision‑making time – and optimise the slower path.

Maintain paired design discussions even if implementation pairing shrinks.

Apply risk‑tiered autonomy models (human‑supervised bots vs. fully autonomous “black‑box factories”).

Shift platform teams from menu‑style tooling to providing the same agent‑driven toolchain they require product teams to use.

Implement red‑yellow‑green risk classification for AI tool usage and continuously scan logs for dangerous patterns.

Treat token and infrastructure spend as governance issues, assigning owners, budget reviews, and usage‑based policies.

Avoid one‑size‑fits‑all AI policies; tier them by system criticality, team maturity, and regulatory exposure.

Protect the human‑centric assets – judgment, aesthetic sense, and deliberate craftsmanship – as strategic differentiators.

Strategic Advice for Executives

Prioritise discipline (testing, governance) before chasing speed gains.

Craft concrete, data‑backed stories for board discussions rather than relying on hype‑driven productivity claims.

Handle token‑budget overruns as governance failures, not just financial surprises.

Adopt risk‑aligned autonomy instead of blanket AI adoption.

Invest in preserving human judgment and design aesthetics as scarce competitive assets.

The summit’s overarching message: as AI makes code generation cheap, the sustainable moat for organizations and engineers is the deliberate preservation of human judgment and craftsmanship.