How a Chinese Bank Leveraged Large‑Model AI to Automate Test Case Generation

Traditional test‑case design at the bank required analysts to read requirement documents, dissect system functions, and manually craft scenarios, a process that was time‑consuming, error‑prone, and heavily dependent on individual experience.

With the rapid advancement of artificial‑intelligence models, the bank explored large‑model capabilities for batch generation of test cases, reducing a multi‑day effort to a matter of hours and improving consistency across testers.

To select a suitable model, the team defined an evaluation scheme covering accuracy, readability, and output stability, and built a benchmark dataset that includes query, addition, deletion, modification, process, and report transaction types, using representative "requirement‑case" Q&A pairs as test baselines. After multiple rounds of testing, a base model was chosen to power the service.

The technical architecture follows an "Engineering + Agent" pattern that fully decouples the engineering unit (handling input/output formatting, requirement parsing, and case presentation) from the AI agent unit (responsible for generation). Interaction occurs through contract‑defined interfaces, allowing independent development and operation of each module.

Despite the benefits, three key challenges emerged: (1) large models struggle with long‑text contexts, causing incomplete requirement decomposition; (2) incompatibilities between the generation service and the existing test‑case management platform required frequent format and protocol adjustments; and (3) the black‑box nature of model outputs made it difficult to trace the logic behind generated cases.

To address these, the bank implemented a customized solution that (1) applies fine‑grained requirement slicing to break complex documents into independent units, mitigating context loss; (2) embeds a rigorous test‑design methodology that maps extracted requirement elements to clear test factors, expected results, and comprehensive factor coverage; and (3) integrates internal R&D template standards so the agent automatically corrects format, logical coherence, and hallucinations.

A multi‑dimensional knowledge base—covering project engineering, business rules, testing experience, design outlines, and historical case assets—feeds a Retrieval‑Augmented Generation (RAG) pipeline. The process, illustrated in Figure 2, splits generation into staged, field‑wise steps, applies dynamic correction, and iterates until the output meets quality thresholds.

The generated cases are managed through the bank’s proprietary test‑management platform “Xingchen”. Figure 3 shows the integration, where the AI engine produces draft cases that can be edited, reviewed, and audited within the platform, while built‑in dashboards track adoption rates, coverage, and modification metrics.

Pilot deployments evaluated the service on various projects. Adoption rates rose from 29.41 % for unified query modules to 38.85 % for open‑payment services. Overall, the average case adoption increased 3.7 ×, with a 28 % adoption rate under a “zero‑modification” condition and 55 % when modifications were allowed.

Having validated the core advantages, the bank plans to scale the service across multiple business lines—including retail, credit, counter services, and risk monitoring—while continuing to reduce demand‑input dependence, curb model hallucinations, and handle increasing business complexity, thereby advancing toward a fully AI‑augmented testing workflow.