How to Build a Complete Prompt Evaluation Pipeline for Reliable AI Outputs

1. Prompt Evaluation Overview

Writing good prompts is only the first step; reliable AI applications require systematic evaluation to verify that prompts perform well across diverse user inputs. Prompt engineering provides techniques for crafting prompts, while prompt evaluation measures their actual effectiveness.

Prompt Engineering vs. Prompt Evaluation

Prompt engineering offers best‑practice tricks such as multi‑sample prompts and XML‑style structuring.

Prompt evaluation automates testing by checking expected answers, comparing prompt versions, and detecting output errors.

Three Paths After Writing a Prompt

Option 1: Test once and assume it’s good – high risk in production.

Option 2: Test a few times and tweak for edge cases – still vulnerable to unexpected inputs.

Option 3: Run the prompt through a full evaluation pipeline, score it objectively, and iterate – requires more effort but yields confidence.

2. Typical Evaluation Workflow

The workflow consists of five key steps, each illustrated with code snippets.

Step 1 – Draft the Prompt

prompt = f"""
请回答用户的问题：
{question}
"""

This baseline prompt serves as the starting point for testing.

Step 2 – Create an Evaluation Dataset

Prepare a JSON array of tasks that represent the kinds of questions the prompt will handle. Example tasks:

"2+2 等于多少？"

"如何制作燕麦粥？"

"月球有多远？"

You can manually write the dataset or ask Claude to generate it.

Step 3 – Process with Claude

Combine each task with the prompt template and send it to Claude:

def run_prompt(test_case):
    """Merge prompt and test case, then return Claude's response"""
    prompt = f"""请解决以下任务：
{test_case[\"task\"]}"""
    messages = []
    add_user_message(messages, prompt)
    output = chat(messages)
    return output

Step 4 – Score the Response

A simple numeric scorer (1‑10) is used as a placeholder. Later sections replace it with model‑based and code‑based scorers.

def run_test_case(test_case):
    output = run_prompt(test_case)
    score = 10  # placeholder
    return {"output": output, "test_case": test_case, "score": score}

Step 5 – Iterate

Run the entire dataset through run_eval, collect results, compute the average score, and refine the prompt based on the metrics.

def run_eval(dataset):
    results = []
    for test_case in dataset:
        result = run_test_case(test_case)
        results.append(result)
    return results

3. Scoring Approaches

Three families of scorers are discussed:

Code scorer – custom logic that checks syntax, length, or presence of keywords.

Model scorer – another AI model evaluates the response and returns a structured JSON with strengths, weaknesses, reasoning, and a numeric score.

Human scorer – manual review for comprehensive quality assessment.

Model Scorer Implementation

def grade_by_model(test_case, output):
    eval_prompt = """
你是一位专业的代码审核专家。评估这个 AI 生成的解决方案。
任务：
{test_case[\"task\"]}
解决方案：
{output}
以结构化 JSON 对象返回 strengths、weaknesses、reasoning、score。只返回 JSON，不要其他内容。
"""
    messages = []
    add_user_message(messages, eval_prompt)
    add_assistant_message(messages, "```json")
    eval_text = chat(messages, stop_sequences=["```"])
    return json.loads(eval_text)

The scorer returns a JSON object like:

{
  "strengths": ["..."],
  "weaknesses": ["..."],
  "reasoning": "...",
  "score": 8
}

Code‑Based Scoring Functions

def validate_json(text):
    try:
        json.loads(text.strip())
        return 10
    except json.JSONDecodeError:
        return 0

def validate_python(text):
    try:
        ast.parse(text.strip())
        return 10
    except SyntaxError:
        return 0

def validate_regex(text):
    try:
        re.compile(text.strip())
        return 10
    except re.error:
        return 0

Each validator returns 10 for valid syntax and 0 otherwise.

4. Dataset Format Enhancements

Include a format field ("python", "json", or "regex") so the pipeline knows which validator to apply. Example entry:

{
  "task": "编写一个 Python 函数，验证 AWS IAM 用户名",
  "format": "python"
}

5. Prompt Clarity Improvements

Explicitly instruct Claude to output only raw code without markdown, comments, or explanations, and use stop sequences to cut off any trailing markers.

prompt = f"""
请解决以下任务：
{test_case[\"task\"]}
严格要求：
* 直接输出代码内容，不要任何其他内容
* 不使用 markdown 代码块
* 不输出注释或解释
"""

6. Merging Scores

Combine model and syntax scores, e.g., by averaging:

model_grade = grade_by_model(test_case, output)
model_score = model_grade["score"]
syntax_score = grade_syntax(output, test_case)
final_score = (model_score + syntax_score) / 2

7. Optimization Exercise

Add a solution_criteria field to the dataset so the model scorer knows the exact evaluation standards. This yields slightly higher and more consistent scores.

8. Quick Quiz

Six multiple‑choice questions reinforce the key lessons about risks of single‑test deployment, choosing fast models for dataset generation, feeding responses to scorers, focusing on evaluation methods, requesting strengths/weaknesses in model scoring, and identifying the model‑based scorer.