Four-Stage Evolution of Intelligent UI Test Case Generation and Execution

Problem Diagnosis

Fast product iteration increases test‑coverage pressure, while complex business rules amplify quality gaps between senior and junior QA engineers. Historical defect knowledge is siloed, PRD changes are frequent, and test‑case writing consumes only 13% of effort but execution consumes 38% of resources.

Four‑Stage Evolution

V1.0 – Prompt Engineering (Feasibility)

Goal: Prove that a large language model (LLM) can generate usable UI test cases from PRD/technical specs.

Approach: Few‑shot learning with 3‑5 high‑quality examples and scenario‑specific prompt templates.

Output: Structured test‑case collections using strict indentation and punctuation for parsability.

Result: Generation rate only 8%; users reported "correct but too generic".

V2.0 – Multi‑Agent Collaboration (Workflow Replication)

Observation: No QA writes test cases in one step; the workflow includes design, review, and iteration.

Design: Three specialized agents – AI generates framework and drafts, humans perform critical review and priority labeling, and a coordinator orchestrates the loop.

Metrics Improvement:

Remaining issue: Generated cases were "correct but shallow".

V3.0 – Knowledge Engineering (Business‑Aware Generation)

Core Idea: Systematically inject business knowledge (textual specs, multimodal assets, historical defect records) so the LLM can reference it during generation.

Components:

Process: When a payment‑related scenario is generated, the system queries the defect library for "concurrent deduction" issues and injects them into the prompt.

Metrics Improvement:

New limitation: Review cost remains high and knowledge maintenance requires manual effort.

V4.0 – Agentic Self‑Evolution (Closed‑Loop Quality Flywheel)

Innovation: Give the system self‑discovery, self‑correction, and self‑evolution capabilities.

Three‑layer capability:

Automated Review & Critique: Instead of binary Yes/No, the system produces structured feedback dimensions (coverage, structure, naming, etc.) and refines incomplete scenarios.

BadCase Loop: QA marks low‑quality cases as BadCase → system extracts failure patterns → builds rule library → automatically applies rules to future generations.

Results:

End‑to‑End Platform (KATE)

Users upload a PRD, the system generates AI‑friendly test cases, executes them with the four‑sense engine, and presents a unified report—all within a single interface.

Key Lessons & Recommendations

Start with high‑value, low‑complexity scenarios (e.g., coverage gaps, vague steps) to quickly achieve >50% generation rate and build confidence.

Prioritize knowledge assets —historical defect library first, then business rule templates, and finally prompt tuning.

Design feedback loops early (user feedback → problem classification → improvement verification) rather than waiting for later versions.

Define clear AI/Human boundaries : AI generates, humans review and correct; this division reduces failure rates and accelerates learning.

Leverage BadCase data as a fuel for continuous self‑evolution.

Conclusion

The four‑stage journey from a feasibility prototype (V1) to a self‑evolving agentic system (V4) demonstrates that AI‑augmented testing succeeds when it combines solid engineering processes, systematic knowledge management, and well‑designed human‑AI collaboration.

输出最终用例集合，关键的三层资产在这里被充分利用：

用户偏好 = {PO 用例置顶，优先级必须标注}

"场景规则模板更新从天级降到 5 分钟，维护成本下降 99%"