20 Loop Design Patterns Every AI Engineer Must Know for Self-Improving Systems
This article details 20 loop design patterns across five categories — quality improvement, memory, planning, exploration, and system optimization — that enable AI systems to iteratively refine outputs, learn from experience, adapt plans, explore alternatives, and optimize their own workflows for production-grade reliability.
Agents vs. Loops
The old way: prompt → response → done. The new way: generate → critique → rewrite → score → retry → remember → improve. An agent is a factory worker who does the job once; a loop is a worker who studies every error, rewrites the playbook, and improves 3% every shift. Teams shipping production AI today aren't writing better prompts — they're building better loops.
Category 1 — Quality Improvement Loops (Make Output Better Before It Leaves)
1. Generate → Critique → Rewrite
The most important loop in AI engineering. Generate output. A critic reviews it. The generator rewrites based on feedback. Repeat until quality threshold is met. Not one model — two roles, one pipeline.
[Generator] → Draft
[Critic] → "Paragraph 3 is vague. Missing evidence. Tone off."
[Generator] → Rewrite based on critique
[Critic] → "Better. But conclusion still weak."
[Generator] → Final rewriteUse for: writing, code review, reports, strategy docs, sales emails. Key insight: the model that generates is not the best judge of its own output. A separate critic catches what the generator misses.
2. Score-and-Retry Loop
Generate. Score. If below threshold, retry. Simple, powerful, underrated.
score = evaluate(output)
while score < threshold:
output = generate(prompt)
score = evaluate(output)
attempts += 1
if attempts > max_retries:
return best_so_farBest for measurable quality: extraction accuracy, format compliance, factual correctness, lead scoring. The generator doesn't know it's being scored; the evaluator does. That separation is the pattern.
3. Multi-Critic Loop
One critic has blind spots. Use four. Each critic evaluates independently. Final output must satisfy all critics before leaving the system. Use for: medical AI, legal document review, financial analysis, regulated content.
4. Adversarial Critic Loop
The critic's only job is to break the answer — not improve it, break it. Adversarial critic asks: What assumptions might not hold? What evidence is missing? What would a skeptic say? Where might this be confident but wrong? Generator then defends or rewrites. Best answers survive attack. Use for: research synthesis, investment thesis review, strategic planning, risk analysis.
5. Judge Ensemble Loop
One judge's score is noisy. Five judges average out noise. Run same output through multiple evaluators. Aggregate scores. Only high-consensus outputs proceed. Use when: single-model evaluation is unreliable, stakes are high, edge cases matter.
Category 2 — Memory Loops (Learn from What Happened So Next Time Is Smarter)
6. Reflexion Loop
The most important self-improvement pattern in existence. Agent fails. Agent analyzes why it failed. Agent stores the lesson. Agent retries with that lesson as context. Every iteration gets smarter.
Attempt 1: Fail
Reflection: "I assumed X, but X was wrong. Next time verify X first."
Attempt 2: Absorb lesson → Partial success
Reflection: "Better. But missed Y. Add Y check."
Attempt 3: SuccessThis is the difference between a system that fails once and a system that never fails the same way twice.
7. Memory Update Loop
After every task completion, store three things: what decision was made, what the outcome was, what you'd do differently next time. Future runs inherit this knowledge. The system at month 6 is not the system at month 1 — it has read 6 months of its own history.
8. Error Library Loop
Store every failure: wrong answers, bad outputs, execution failures, edge cases. Before a new task, search the error library. If similar failure exists → apply known fix before starting. System never trips on the same error twice. Most underrated pattern in production AI.
9. Success Pattern Loop
Most engineers only store failures. Store wins too. When a task goes well: save the approach, save the context, save what made it work. On similar tasks, retrieve success patterns. Learn from victories, not just mistakes.
10. Memory Compression Loop
Memory grows unbounded. Unbounded memory is unusable memory. After N memories: compress them. Many concrete memories → fewer high-level abstractions.
Before compression:
"Failed on Task A because X"
"Failed on Task B because X"
"Failed on Task C because X"
After compression:
"Pattern: X causes failure. Always check X first."Context stays bounded. Patterns stay accessible. System stays fast.
Category 3 — Planning Loops (Adapt the Plan When Reality Changes)
11. Plan → Execute → Replan
Biggest mistake in agent design: treating the plan as fixed. Plans survive first contact with reality. Pattern: Plan → Execute step → Observe result → Update plan → Continue. Not waterfall. A spiral. Each turn tightens the approach. Use for: changing environments, interdependent tasks, long-horizon goals.
12. Dynamic Workflow Loop
Most pipelines are fixed: Step 1 → Step 2 → Step 3. Forever. Dynamic workflows change based on results. If Output A → run Branch X. If Output B → run Branch Y. If Output C → jump to Step 5. Pipeline decides its own shape at runtime. Use for: multi-document research, customer support routing, adaptive content pipelines.
13. Goal Decomposition Loop
Big goal enters system. System breaks into sub-goals. Each sub-goal into tasks. Each task into steps. Keep decomposing until every unit is small enough to execute in one call.
Goal: "Write comprehensive competitive analysis"
↓
Sub-goal 1: "Identify top 5 competitors"
Sub-goal 2: "Analyze each competitor's product"
Sub-goal 3: "Compare pricing models"
Sub-goal 4: "Identify gaps"
↓
Each sub-goal → Tasks → Single model callsLoop keeps decomposing until system can act.
14. Progress Assessment Loop
Every N steps: stop and ask, "Are we actually closer to the goal?" If yes: continue current strategy. If no: change strategy, tools, or plan. System monitors its own progress instead of blindly executing. Use for: long-running research agents, multi-day autonomous tasks, debugging agents.
15. Constraint Satisfaction Loop
Keep running until all constraints are satisfied.
while not all_constraints_satisfied(output):
output = improve(output, unsatisfied_constraints)
constraints = [
budget_under_limit,
quality_above_threshold,
latency_under_200ms,
tone_matches_brand,
no_hallucinations
]Common in production systems. Output isn't done until every business rule passes.
Category 4 — Exploration Loops (Find the Best Answer by Trying Multiple Paths)
16. Branching Exploration Loop
Don't bet on one path. Explore multiple simultaneously.
paths = [
generate(approach="conservative"),
generate(approach="aggressive"),
generate(approach="creative")
]
scores = [evaluate(p) for p in paths]
best = paths[scores.index(max(scores))]Compare results. Pick best branch. Discard rest. Use for: content variants, architecture decisions, multi-hypothesis debugging, A/B generation.
17. Tree Search Loop
Branching goes one level deep. Tree search goes as deep as needed. Expand most promising nodes. Prune weakest. Keep exploring until solution found.
root → [A, B, C]
A → [A1, A2] # A looks promising, expand it
B → prune # B is weak, stop here
A1 → [A1a, A1b]
A1a → solution ✓Use for: complex reasoning chains, multi-step planning, code debugging, research synthesis. Computationally expensive but finds solutions single-pass cannot.
18. Debate Loop
Two agents. One topic. Opposite positions. Agent A argues for answer. Agent B argues against. Each round challenges assumptions, demands evidence, exposes weak logic. Final answer emerges through disagreement, not consensus. Adversarial pressure finds what confident single-agent answers miss. Use for: investment decisions, strategic planning, risk assessment, research critique.
Category 5 — System Optimization Loops (Loops That Improve the Loops)
19. Prompt Optimization Loop
Most engineers write a prompt once and never touch it again. Prompt optimization loop changes that. System: runs prompt on test set, evaluates outputs, identifies failures, rewrites prompt based on failures. Prompts get better automatically. No human touch needed.
current_prompt = "Summarize this document."
for iteration in range(max_iterations):
outputs = [run(current_prompt, doc) for doc in test_set]
scores = [evaluate(o) for o in outputs]
avg_score = mean(scores)
if avg_score >= target:
break
failures = [o for o, s in zip(outputs, scores) if s < threshold]
current_prompt = improve_prompt(current_prompt, failures)
# Prompt rewrites itself based on failuresUse for: production pipelines, automated content systems, classification tasks. Best prompts in production aren't written by humans — they're evolved.
20. Workflow Optimization Loop
This is where it gets real. Loops improving loops. System measures own performance: then modifies own workflow. Too slow? Parallelize two steps. Too expensive? Replace one GPT-4 call with smaller model where quality unaffected. Quality dropping? Add critic before final output.
metrics = measure_workflow(outputs, latency, cost)
if metrics.latency > target_latency:
workflow = parallelize(slow_steps)
if metrics.cost > budget:
workflow = replace_with_cheaper_model(high_cost_steps)
if metrics.quality < threshold:
workflow = add_critic_before(final_output_step)Real self-improving systems start here. Not just output improving — the system redesigns itself.
The Meta-Pattern Behind All 20 Loops
Every loop above shares the same structure: Act → Observe → Evaluate → Adjust. That's the complete recipe. Output is never final on first try. Output is just a starting point. The loop is what turns a starting point into a production-grade result.
Complete Map
Category 1 — Quality Loops (Make it better before it leaves)
1. Generate → Critique → Rewrite
2. Score and Retry
3. Multi-Critic
4. Adversarial Critic
5. Judge Ensemble
Category 2 — Memory Loops (Learn from what happened)
6. Reflexion
7. Memory Update
8. Error Library
9. Success Patterns
10. Memory Compression
Category 3 — Planning Loops (Adapt when reality changes)
11. Plan → Execute → Replan
12. Dynamic Workflow
13. Goal Decomposition
14. Progress Assessment
15. Constraint Satisfaction
Category 4 — Exploration Loops (Find best answer by trying multiple paths)
16. Branching Exploration
17. Tree Search
18. Debate
Category 5 — System Optimization Loops (Loops that improve loops)
19. Prompt Optimization
20. Workflow Optimization
Most Engineers Think Agents Are the Future
Agents are just workers. Loops are what make workers improve. The biggest shift in AI right now isn't better models. It's the shift from: Prompt → Response to: Generate → Evaluate → Learn → Improve . Teams that master loop design won't just build better prompts. They'll build systems that get better every day after deployment — without anyone touching them.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
