Loop Engineering: Building Reliable Automated Agent Workflows

Concept Definition: What Is a Loop?

Loop engineering moves beyond single-turn prompting. Instead of manually driving each interaction, engineers design a small system that repeatedly: reads external state (CI, issues, PRs, logs), decides the next task, executes via an agent, verifies results (tests, reproduction, scans), writes state outside the conversation, and evaluates stop conditions (continue, retry, escalate, end). Missing step 5 makes it a mere session; missing step 6 turns it into a token-burning timer.

Addy Osmani positions loop engineering above the agent harness. The harness (guides and sensors) controls the agent's environment and self-correction; the loop handles scheduling, context, task selection, verification, state storage, and termination. Birgitta Böckeler's harness engineering splits control into pre-execution guides (rules, skills, context) and post-execution sensors (tests, lint, review agents). Without the loop, harnesses remain one-off tools; without harnesses, loops become scheduled mistake generators.

Research Background: Why Loop Now?

Product capabilities are converging. Codex exposes automations, worktrees, skills, plugins, MCP, subagents, memories, and Goal mode. Claude Code adds scheduled tasks, /loop, goals, hooks, skills, subagents, worktrees, and MCP. OpenAI's Cookbook describes an agent improvement loop using real traces, human/model feedback, reusable evals, handoff artifacts, and Codex-driven harness updates. Amplitude's Ralph loop connects product data, opportunity generation, browser verification, and coding agents—each cycle builds an opportunity, verifies via browser, instruments new features, and only auto-merges low-risk changes. Recent papers ( Engineering Robustness into Personal Agents with the AI Workflow Store and EurekAgent ) argue that on-the-fly loops skip essential software engineering steps; reliability comes from reusable, constrained workflows with explicit permissions, artifacts, budgets, and human-in-the-loop gates.

System Composition: Six Components

4.1 Automations: Heartbeat, Not Brain

Automations wake the loop. Codex automations run on schedules, feed results to a triage inbox, and can auto-archive. Claude Code's scheduled tasks and /loop suit intra-session checks (deployment status, PR comments, CI). Hooks provide deterministic lifecycle guards: format on edit, block sensitive files, notify on input, reinject context. A good automation prompt reads like an operating procedure, not a wish list:

Read failed CI from last 24 hours.
Only process failures related to current PR.
If failure is infra flake, record and do not modify code.
If reproducible, create minimal fix.
Max 3 files changed per run.
Write results to loop-state.md after tests pass.
Stop and escalate on permission, migration, or security changes.

Bad: "Check project daily, fix any issues." That delegates judgment to the model.

4.2 Worktrees: Isolate Before Parallelizing

Git worktrees give each task an independent checkout, preventing file conflicts. Codex and Claude Code both support worktrees for parallel sessions. However, worktrees don't solve review bandwidth. If eight agents open PRs daily but you can only deeply review two, throughput shifts from writing to understanding. Design principle: parallelize only low-coupling tasks; merge rate limited by human comprehension.

4.3 Skills: Externalize Project Knowledge

Without skills, every loop restart is like a new hire guessing test commands, directory layout, and team conventions. Skills (Codex's SKILL.md, Claude's skills) move stable knowledge out of context. Suitable content: test/lint/typecheck commands, protected directories, migration requirements, PR description templates, flaky test classification, historical incident patterns. Keep skills small and focused (e.g., triage-ci, fix-flaky-test, review-pr, frontend-qa, security-check). Descriptions must be explicit because agents decide whether to load a skill based on its description. Benefits: saves tokens and prevents dangerous defaults from open-source patterns.

4.4 Plugins and Connectors: Connect Loop to Real Tools

Real engineering state lives in GitHub, Linear, Slack, Sentry, databases, staging APIs, browsers, CI. Three layers: MCP (protocol), Connectors (specific integrations), Plugins (distribution bundles). Conflating them misleads teams into thinking installation equals full control. A loop that reaches external tools can:

Detect CI failure
Read failure logs
Locate related commit
Fix in worktree
Run tests
Open PR
Link Linear ticket
Notify Slack
Write result to state file

Permission layers by risk: observe (read-only), propose (generate patches/PRs), act with approval (external actions with human sign-off), autonomous (only low-risk, reversible, well-verified tasks). Most teams should solidify observe and propose first.

4.5 Sub-agents: Separate Maker and Checker

Writing and reviewing agents must differ. The implementer easily rationalizes its own output. Codex subagents move exploration, testing, log analysis out of the main thread to avoid context pollution. Practical split: explorer (read-only diagnosis), implementer (minimal changes), verifier (reproduce, test, acceptance), reviewer (diff analysis for boundaries, regressions, over-engineering), security reviewer (permissions, data exposure, injection, dependencies, secrets). Not every task needs five agents; sub-agents add token and coordination overhead. Use for verification stages and parallel read-heavy tasks. Small tasks: implementer + verifier often suffice.

4.6 Memory: State Must Live Outside Conversation

Long-running loops cannot rely on chat history—context compresses, threads break, models forget, agents repeat dead ends. External state can be a simple loop-state.md recording: current goal, handled items, attempted approaches, failure reasons, passed verifications, human-judgment items, next entry point. Can also be Linear board, GitHub labels, database, PR comments, or Codex memories. Memories suit stable preferences and conventions; hard constraints belong in repo docs ( AGENTS.md or skills) for reviewability, versioning, sharing. Principle: models may forget, system state must not.

Method Examples: Real-World Loops

Start with low-risk loops, not "fully autonomous engineering teams."

CI Triage Loop

Run daily at 9:00.
Read failed CI from last 24 hours.
Classify: real regression, flaky, infra, unknown.
Only handle real regressions affecting current PR.
One issue per run.
Write reproduction steps before code change.
Run corresponding tests after fix.
If tests pass, open/update PR.
If tests fail, record failure reason.
Stop on migration, security, permission, billing, production config changes.

Unsexy but deployable: clear boundaries, stable inputs, reviewable outputs, explainable failures.

Review Comment Loop

Periodically read unresolved PR review comments.
Categorize: mechanical fixes, needs judgment, cannot handle.
Apply mechanical fixes directly.
Summarize judgment-needed comments into issue list.
Explain unhandled comments.
Run minimal relevant tests after each change.

More reliable than "optimize my project" because goals and judgments aren't left to model improvisation.

Harness Maintenance Loop

Weekly: read recently merged agent PRs and review comments.
Identify recurring failure patterns: wrong public API changes, missing migrations, happy-path-only tests, over-refactoring.
For deterministic catches, add lint, type checks, architectural rules, test fixtures first.
For semantic judgments, update review skill or verifier prompt.
Change one harness control per iteration.
Record what the control caught and any false positives.

This loop doesn't ship features directly but makes subsequent feature loops more stable. Teams often chase "agent writes more code" before asking "how to stop agent repeating same mistakes."

Automation Criteria: When to Loop

Five filters plus a sixth:

Stable input? (CI, issues, review comments, logs — yes; vague strategy — no)

Reviewable output? (PR, report, state file, ticket — yes; implicit changes — no)

Explicit verification? (tests, reproduction steps, schema checks, browser paths — yes; "looks better" — no)

Controllable permissions? (default read/propose, gradually expand execution)

Recoverable failure? (worst case: a pending patch, not a broken shared environment)

Does the loop leave reusable artifacts? (better fixtures, clearer rules, smaller skills, stabler scripts, reusable workflows). If not, you're just re-buying the same improvisation.

If criteria aren't met, clarify the process first.

Risk Analysis: Loop Failure Modes

7.1 Overly Coarse Objective

"Improve quality," "optimize experience" are not valid goals. Agents translate them into plausible but irrelevant actions (refactor, rewrite copy, add tests, delete duplication). Goals must be verifiable conditions:

Make the 3 failing cases in checkout.spec.ts pass, without modifying test assertions or production logic outside the payment gateway mock.

7.2 Verification Swallowed by Agent

Agents summarize long logs as "mostly passed, few unrelated issues." In a loop, that must count as failure. Verification results must be structured: command, exit code, failing cases, stop-condition satisfaction.

7.3 State File Unread

Writing state is step one; next prompt must explicitly require reading it and deciding continuation. Otherwise it's an unread daily report.

7.4 Parallelism Creates Understanding Debt

Parallel agents create illusion of doubled throughput. The real question: do you still understand the merged system? If you only read summaries, debt accumulates fast.

7.5 Unbounded Cost

Token cost is an architectural concern. Every loop needs a budget: max attempts per run, max PRs per day, sub-agent eligibility, stall timeout, mandatory human handoff triggers.

Engineering Responsibility: What Remains for Engineers

Loops don't absolve responsibility; they relocate it to where cutting corners is harder. Engineers still decide: what to automate, which permissions to grant, what state to persist, which diffs require human review. Stop conditions cannot be left to model improvisation. Tasks with unclear stop conditions shouldn't run unattended.

"Hand to AI" writes: Help me maintain the project. Loop engineering writes:

Check current PR's CI every weekday morning.
Only handle stably reproducible test failures.
First locate failing test and related diff.
Modification scope limited to failure-related modules.
Run corresponding tests.
Write command, result, next step to loop-state.md.
If same failure reason occurs twice consecutively, stop and escalate to me.

The first buys comfort; the second designs a system.

Conclusion

Loop engineering is not a repackaged prompt trick. It's closer to engineering process design: rhythm, context, tools, permissions, verification, memory, stop conditions. It may become the primary way coding agents are used—not because agents are reliable enough to run unmanaged, but because single conversations cannot carry complex engineering work. Once agents span multiple turns on real tasks, state, feedback, and responsibility boundaries become unavoidable.

Direct prompting remains useful; many tasks belong in one conversation. Don't force everything into loops for appearances. For recurring, well-bounded, verifiable, controllable engineering tasks, loops are worth building.

Final litmus test: does this loop deepen your understanding of the system, or does it let you read less code, judge less, and carry less responsibility? The former is leverage; the latter is just faster debt accumulation.