Why AI Product Evaluation Is Hard and How to Build a Scientific Assessment Framework

1. Core Challenges of AI Product Evaluation

Challenge 1: Output Uncertainty – Traditional products produce deterministic results (same input → same output), while AI products may generate different outputs for the same input, making deterministic rules ineffective.

Challenge 2: Predominantly Subjective Questions – Traditional testing uses binary pass/fail cases, whereas AI testing often involves essay‑type questions (e.g., quality of generated text) that lack a single correct answer and require multi‑dimensional assessment.

Challenge 3: Over‑fitting Risk – When engineers build the test set, they may inadvertently create an “open‑book” scenario where the model memorizes the cases, achieving 100% pass rate offline but failing in production.

Challenge 4: High Evaluation Cost – Automated tests for traditional software run in minutes; AI tests often need manual review, taking minutes per case.

Challenge 5: Ambiguous Metrics – Traditional products are judged by functional correctness, while AI products must be measured by user‑perceived value, requiring scientific indicators.

2. AI Product Evaluation vs. Traditional Testing

Traditional testing relies on deterministic inputs and objective pass/fail criteria. AI testing must handle stochastic outputs, subjective scoring, and value‑oriented metrics.

3. Who Should Build the Test Set?

Product Manager – defines evaluation standards and ensures the test set reflects user value.

Business Experts – provide realistic scenarios and user feedback.

Technical Staff – execute the tests and supply technical support.

Core principle: the team closest to the user should own the test set.

4. Core Evaluation Process

Step 1: Build Evaluation Plan – Clarify goals, scope, and resources. Step 2: Construct Test Set – Collect, write, and review use cases. Step 3: Define Evaluation Rules – Set pass criteria, dimensions, and scoring methods. Step 4: Execute Evaluation – Conduct manual testing, automated testing, and intelligent‑assistant testing. Step 5: Assess Results – Compute metrics, analyze outcomes, and decide on launch.

Step 1 – Evaluation Plan Template

Goal: Verify AI dialogue generation meets launch standards.

Scope: Scenarios covering first visit, objection handling, deal closing.

Roles: New reps, senior reps.

Products: Product A, Product B, competitor comparison.

Resources: 1 PM, 2 business experts, 3 days, 200 cases.

Launch thresholds: Pass rate >70%, efficiency gain >20%, user satisfaction >4.0.

Step 2 – Test Set Construction

Examples of case distribution and sample case images are shown below.

Each case includes fields such as scenario, user role, difficulty, and expected dimensions (correctness, completeness, tone, conciseness, usefulness).

Step 3 – Evaluation Rules

Objective questions require exact or semantic matches. Subjective questions are scored on a 1‑5 scale across multiple dimensions, with a passing average ≥4 and no dimension below 3.

Step 4 – Execution

White‑box testing : First 100 cases known to developers for rapid bug fixing.

Black‑box testing : Last 100 cases hidden from developers; only pass rate is reported.

Evaluators : Product manager + 2 business experts assess correctness, completeness, tone, conciseness, and usefulness.

Step 5 – Metrics and Decision

Pass Rate = Passed cases / Total cases Efficiency Gain = (Manual time – AI time) / Manual time User Satisfaction = Simulated user rating (1‑5) Decision : If overall pass rate >70% and efficiency gain >20%, the feature can be launched; otherwise iterate.

5. Real‑World Case: AI Sales Script Generation

Product : AI‑driven sales script generator for 2000+ medical representatives.

Goal: Verify whether the AI meets a >70% pass rate, >20% efficiency gain, and >4.0 user satisfaction.

Resources: 1 PM, 2 business experts, 3 days, 200 test cases.

Results :

Overall pass rate 70% > threshold 50% → launchable.

Efficiency gain 20% → meets standard.

Simple scenarios (first visit, relationship maintenance) performed well; complex scenarios (objection handling, deal closing) need improvement.

Cost‑saving calculation example: 20% efficiency per case, 60 s manual time, 100 uses per person per day, 100 users → 33 hours saved daily, equivalent to 8,000 CNY.

6. Five Key Success Factors

Product‑Manager‑Led – ensures the test set reflects user value and balances feasibility.

Continuous Iteration – launch with a minimal viable test set (≈50‑100 cases) and refine based on feedback.

Even Scenario Coverage – distribute cases across scenarios, user roles, and difficulty levels.

Automation + Intelligent Assistance – use automated tests for objective items and large‑model assistants for subjective scoring, keeping critical cases manual.

From Offline to AB Test – after offline validation, run small‑scale gray releases, then full AB tests measuring adoption, efficiency, satisfaction, and renewal rates.

7. Future Trends

Widespread use of large‑model assistants for subjective evaluation, reducing cost.

Higher automation ratio for both objective and subjective items.

Industry‑wide shared test sets enabling benchmark comparisons.

In summary, AI product evaluation requires a dedicated, user‑value‑driven test set, scientific metrics, iterative development, a mix of manual, automated, and AI‑assisted testing, and a clear path from offline validation to AB testing to ensure reliable launch decisions.