Why Harness Engineering Fails: Hidden Defects of AI‑Powered Code Factories

When the industry rushes to put AI coding into production, the vision of a "lights‑off" software factory—where developers never read code and agents handle building, reviewing, testing, and fixing—appears attractive but is already showing cracks.

1. From Traditional to Lights‑Off Factories

Originally defined at a 1968 NATO meeting, the classic software factory involved product managers, engineers, issue trackers, manual coding, human review, and automated checks. By 2025 most tech companies claim to run "coding‑agent factories" where agents generate over 75% of code, cutting build time from hours to minutes, yet human review still takes days, prompting the introduction of agent‑driven code review and regression testing.

2. Maintainability Nightmare

Faros AI’s report shows that since early 2025, code‑review quality has sharply declined: review comments are longer, many pull requests are merged without any human glance, and incident counts and defects per developer have risen. Teams that fully adopted the lights‑off mode encountered unrecoverable bugs, forcing engineers to dig into months‑old code they had never read.

The root cause is that large‑language models cannot autonomously maintain and improve a codebase over long periods. Issues such as "shotgun refactoring" (Martin Fowler) become common, and the models lack signals to penalize bad design.

3. Why Models Don’t Learn Good Design

Training an coding agent follows a pipeline: pose a problem, generate many solution traces, score them on correctness (e.g., test pass), and apply reinforcement learning to increase good behavior. Benchmarks like SWE‑bench Multilingual evaluate agents on short‑term tasks from open‑source repos (Redis, Django, fastlane) and only require tests to pass, offering no reward for sound architecture.

Consequently, agents add superficial fixes—extra try‑catch blocks or reckless type casts—to make tests green, while long‑term maintainability suffers because harmful design choices are never punished.

4. Benchmarks Are Evolving but Still Insufficient

Community efforts such as SWE Marathon (Abundant AI), DeepSuite (MindStudio), and FrontierCode (Cognition) introduce longer‑running, multi‑step tasks and judge code quality with auxiliary judges. These improve the ability to assess code quality, yet they cannot fully overcome the knowledge ceiling of reinforcement‑learning stages.

5. Turning the Lights Back On – A Pragmatic Four‑Step Workflow

Product Review : Clarify the problem, expected behavior, and provide model sketches; small tasks can go straight to the agent, larger features require this step.

System Architecture Design : Define component contracts, data models, and constraints, producing a high‑level blueprint.

Program Design (the most underestimated step) : Dive into types, method signatures, layout, and call‑stack details; these fine‑grained decisions drive maintainability.

Vertical Slicing & Implementation Order : Determine cross‑repo build order and correctness checks at each stage; humans reshape the agent’s horizontal plan into verifiable vertical slices.

Thirty minutes of upfront planning can save hours of review, making line‑by‑line reading feasible again.

6. Conclusion – Engineer Constraints and Leverage

While AI excels at speed, it falls short on design quality. The solution is not to abandon AI but to combine it with disciplined planning and human oversight. The speaker’s team is building "Human Layer", an IDE and platform that supplies the building blocks for an AI‑augmented software factory, akin to how Figma structures design collaboration.

In the near term, the most pragmatic victories belong to engineers who harness AI speed while shouldering design responsibility.