GEPA: Zero-Config Prompt Self-Evolution via Reflective Mutation & Pareto Optimization

Background: Why Automate Prompt Optimization?

Current LLM Judge and agent prompt tuning relies on manual experience-driven iteration , suffering from four core problems: (1) High blindness — changes lack data support and quantitative verification; (2) Pseudo-automation — static scans by stronger models lack deep self-reflection loops; (3) Hard to replicate — expertise stays with individuals, no standardized process; (4) No trade-off mechanism — single-metric fixes cause adversarial degradation (e.g., precision up, recall down) without a Pareto frontier to preserve balanced candidates.

In production, LLM Judges face two alignment targets: offline alignment with human annotations (ground truth consistency) and online alignment with business metrics (CTR, conversion). Manual tuning converges slowly, cannot bridge deep semantic gaps, and fails to generalize across domains.

Technical Foundation: GEPA Paper & DSPy Ecosystem

The solution builds on the ICLR 2026 paper GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning (arXiv:2507.19457, https://arxiv.org/abs/2507.19457), which introduces Reflective Mutation : LLM automatically analyzes bad cases → attributes errors to specific prompt rules → generates candidate variants → selects via Pareto frontier multi-objective optimization. GEPA belongs to Stanford's DSPy (Declarative Self-improving Python) ecosystem (arXiv:2310.03714, https://arxiv.org/abs/2310.03714), shifting prompt engineering from craft to algorithmic optimization.

System Architecture: Four-Layer Design

3.1 Main Loop: Inference → Scoring → Reflection → Selection

The GEPA optimization loop comprises four stages:

Seed Prompt — user-provided or Inspector-generated initial prompt.

LLM Inference — task model (temp=0.0) runs batch inference on training set, producing raw outputs.

Scorer — parses structured results (JSON field / regex) and computes task-specific metrics: Precision/Recall/F1 for binary, Cohen's Kappa/Spearman/MAE for scoring, Pair F1/ARI for clustering, etc.

Reflection & Mutation — two-step: (a) Error Attribution via Judge model (three-dimension: error location, prompt rule attribution, improvement suggestion); (b) Prompt Mutation by Reflection model (temp=0.7, stronger than task model) consuming reflection tuples (input, output, score, feedback) to generate improved candidates, retained via Pareto frontier.

Pareto Frontier Selection : Instead of picking a single "best" candidate, GEPA keeps all non-dominated candidates (e.g., high precision/low recall vs. low precision/high recall), preserving diversity and avoiding local optima.

3.2 Three Model Roles

Task Model (temp=0.0): executes the task (classify/score/cluster), fast deterministic output.

Judge Model : analyzes errors — location, prompt attribution, fix suggestion — providing objective third-party view.

Reflection Model (temp=0.7, stronger): synthesizes all feedback into improved prompt candidates, encouraging creative mutation.

3.3 Five-Step Workflow (v2.0 Config-Driven)

Collect Info — annotated dataset + one-sentence task goal.

Analyze Data — Inspector auto-infers task type, fields, templates, seed prompt.

Confirm Config — adjust scoring method, split ratios.

Run Optimization — background 15–30 min, real-time JSONL progress.

Report Results — score lift, optimized prompt, diff vs. original, next-step suggestions.

3.4 Code Architecture (Layered)

The codebase is organized into four layers:

Config Layer — defines all configs via Pydantic; build_config() aligns output schema, extraction, scoring by task type. Key files: config.py, config_factory.py.

Engine Layer — exposes run_optimize, run_evaluate, validate_config; orchestrates data, adapter, models. Key files: engine.py, adapter.py ( GenericGEPAAdapter), run.py.

Components Layer — pluggable modules via Protocols: parsing, scoring, feedback, LLM client, evaluation, inspection. Key files: parsers.py, scorers.py, evaluate.py, feedback.py, llm.py, inspector.py, protocols.py.

Service Layer — CLI, FastAPI HTTP, Claude Code Skill; job state persisted to filesystem. Key files: api.py, jobs.py, progress.py.

Key Component Details

4.1 Config Layer: Safe Configuration Builder

TaskConfig

aggregates sub-configs (Model, Data, Prompt, Input, Output, Scoring, Feedback, Optimization, Evaluation). build_config() in config_factory.py auto-aligns three coupled pieces — output schema, answer extraction, scoring function — by task-type presets, eliminating manual mismatch errors.

4.2 Engine Layer: Running the Task

engine.py

exposes three APIs. run_optimize prepares train/val/holdout splits, assembles GenericGEPAAdapter, builds reflection model (temp=0.7), sets stop conditions (no-improvement streak / target score), runs GEPA main loop, saves best prompt, re-evaluates on holdout for generalization. GenericGEPAAdapter translates one candidate evaluation into: batch inference → output parsing → batch scoring → reflection trajectory generation (per-sample score + feedback).

4.3 Component Layer: Six Pluggable Modules

data.py — load (json/jsonl/csv) → strict label validation → stratified split (50/30/20) → template formatting → GEPA DataInst.

inspector.py — zero-config inference: profiles each field (type, cardinality, length, missingness) → infers label field, task type, input fields → outputs DatasetProfile with suggested config, seed prompt, enum values, score ranges.

llm.py — unified multi-provider (DashScope/OpenAI/Anthropic/Moonshot/Zhipu) via litellm; batch concurrency, exponential backoff retry (2s/4s/8s, max 3), per-request structured success/error, no silent failures.

parsers.py — cleans raw text, extracts via json_field / regex / full_text /custom; validates against OutputSchema (types, enums, numeric bounds); build_format_instruction() generates format spec from schema, appends to system prompt — ensuring required format and validated format stay in sync.

scorers.py + evaluate.py — two-level scoring: sample-level (exact_match, contains, numerical_closeness linear/quadratic, weighted_binary asymmetric FP/FN) feeds reflection trajectories; batch-level (F1 balanced, precision_recall constrained — returns recall only if precision ≥ target else penalizes by (precision/target)²) serves as GEPA objective. evaluate.py auto-selects metric suites per task type.

feedback.py — dual-layer: Rule layer (template-based factual feedback: correct/parse-fail/wrong with model vs. gold); Deep layer (Judge model on error samples only, three-question attribution: error location, prompt rule cause, improvement suggestion). The second question — mapping "model got it wrong" to "which prompt rule failed" — turns blind search into directed rewrite.

protocols.py — Python Protocol + ABC base classes for DataPreparer, OutputParser, Scorer, FeedbackGenerator; users implement and register via config class path without touching engine code.

4.4 Service Layer: Three Entry Points

CLI — local debugging.

HTTP API ( api.py, FastAPI) — task-centric endpoints: submit, list, status, progress, result, cancel, one-off evaluate, validate, upload, health.

Claude Code / Qoder Skill — conversational five-step flow.

Job state persisted to runs/jobs/{job_id}/ with five files: config.json, job_state.json (pending/running/completed/failed/cancelled), progress.jsonl (real-time events), best_prompt.txt, result.json. Filesystem persistence enables background threads, cancellation tokens, and post-hoc audit.

Usage & Results

Inputs: annotated JSONL dataset (input fields + answer field + optional image URL fields), optional seed prompt ( system_prompt.txt + user_template.txt with {fieldName} placeholders). Interactive prompts collect dataset path, task goal, existing prompt path, judgment preference (precision vs. recall vs. F1 vs. precision-target), optional model name, input fields, multimodal flag.

Outputs in run_dir: best_prompt.txt, optimize_result.json, candidates.json, candidate_tree.html (visualization), progress.jsonl, logs, engine snapshot, per-sample validation outputs. Chat report shows validation/test metrics, key prompt diffs, and next-step recommendations.

Scoring Preference Cheat Sheet

Maps natural-language preferences to scorer configs:

"don't false positive" → weighted_binary with {fp_penalty:0.0, fn_penalty:0.5} "precision ≥ 80% then maximize recall" → precision_recall with {precision_target:0.8} Scoring tasks choose linear vs. quadratic penalty

Comparison tasks toggle check_bias Clustering tasks select Cohen's Kappa/ARI or pair precision/recall focus

Future Directions & Open Challenges

6.1 Frozen Zones: Not Everything Should Mutate

GEPA by default rewrites the entire prompt, risking reward hacking — dropping hard constraints (tool protocols, output schema) can boost scores artificially. Defense: structurally split prompt into Frozen Zone (hard constraints, tool protocols, output schema — never mutated) and Mutable Zone (judgment priorities, decision tendencies, exception handling, phrasing). Multi-component GEPA structure lets selector return only mutable components.

6.2 Prompt Bloat: Target "More Accurate, Not Longer"

Reflection models naturally add explanations; over 10+ iterations prompts grow from hundreds to thousands of tokens. Hidden costs: key constraints buried, attention diluted; inference cost & latency double; human unreadable — impossible to debug or rollback. Mitigation: add prompt length as third Pareto objective (not weighted penalty, to avoid canceling primary goal) with hard length gate.

6.3 Prompt Self-Evolution Without Ground Truth

Highest-value targets are often evaluated agents (policy generation) with no correct answer, only downstream business metrics. Two ideas: (1) Mutation Direction — augment native GEPA's random exploration with textual gradients from offline evaluator / online business attribution + prompt structure defect diagnosis, translating "where business fails" into "which prompt segment to change". (2) Correctness Criteria — (a) use existing offline metrics as judge, construct adversarial dual objectives (missed actions / spurious actions); (b) historical policy replay: trace candidate decisions to similar historical scenarios, compare against best historical business metrics. Replay used only for ranking/veto, not fed back into reflection, else "change less → match more → score higher" drives search toward conservatism.

References

[1] AGRAWAL L A, TAN S, SOYLU D, et al. GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning. arXiv:2507.19457, 2025. URL: https://arxiv.org/abs/2507.19457

[2] KHATTAB O, SINGHVI A, MAHESHWARI P, et al. DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines. arXiv:2310.03714, 2023. URL: https://arxiv.org/abs/2310.03714