How AI Agents Boost Test Design Quality and Efficiency: Innovative Practices and a Systematic Framework

1. Testing Challenges at Million‑Scale Asset

When the test asset library of China Industrial and Commercial Bank’s software development center exceeded one million items, traditional testing struggled with resource‑demand mismatches, knowledge transfer gaps, and difficulty retrieving relevant cases during iterative regression, making test design a bottleneck.

2. Intelligent Testing Hub: Two‑Stage Generation and Retrieval

The center adopted a data‑driven, AI‑empowered methodology that spans the entire test lifecycle. By mining requirement documents, functional designs, and other multidimensional data, it built a dual‑stage mechanism—intelligent case generation before test admission and precise retrieval during test design—forming a closed‑loop capability matrix.

3. Building the Test Asset Knowledge Base

Intelligent document splitting : Vectorize each test case and split files line‑by‑line to keep whole cases intact, avoiding mixed or incomplete retrieval results.

File tagging governance : Tag cases by category so that retrieval can filter precisely according to the specified label.

bge‑reranker‑v2‑m3 model selection : Combine keyword and vector search for initial recall, then re‑rank with the bge‑reranker model using a custom cosine‑similarity threshold to output the most relevant top cases.

Automated knowledge‑base updates : Weekly automatic pipelines pull new cases, structure them, vectorize them, and incrementally update the knowledge base, keeping it synchronized with source systems.

4. Smart Pre‑Generation in the Requirement‑Testing Phase

After developers finish functional design, a large language model parses requirement documents and designs, extracts entities and relations (test objects, steps, expected results), and automatically generates standard test cases. The generated cases are then hybrid‑recommended together with similar existing assets from the knowledge base.

5. Dynamic Optimization in the Test Design Phase

Incremental case matching and reuse : The system intelligently generates complementary cases for newly added scenarios and uses a hybrid retrieval strategy to replace redundant manual entries with suitable existing cases.

Automated asset linking and freshness : An intelligent mapping links test cases to automation scripts, detects script validity, and assists testers in diagnosing and updating scripts, ensuring scripts evolve together with business changes.

6. High‑Quality Asset Library Construction

A multi‑dimensional quality assessment model with 14 metrics (completeness, clarity, etc.) evaluates all cases. Existing cases receive full‑scale scoring, while new cases are assessed daily. Scores are refined through LLM‑based semantic analysis, and low‑quality cases are automatically suggested for improvement and replaced with standardized high‑quality examples.

7. Application Results

The AI‑driven transformation shortened test‑case design cycles by 65%, raised asset reuse to over 59%, and delivered a robust, continuously improving test asset pool that supports large‑scale regression with high precision.

8. Future Outlook

With deeper AI integration, the testing process will shift from manual‑centric to intelligent‑collaborative, maintaining system quality while dramatically boosting delivery speed, thereby strengthening financial digital transformation.