TestAgent: Open-Source 7B LLM for Multi-Language Test Generation

What is TestAgent?

TestAgent aims to build a testing‑domain "agent" by combining large‑model capabilities with engineering techniques, enabling a 24‑hour online testing assistant that makes testing smooth and efficient.

We are excited to announce the first domestic open‑source test‑industry large model and tools – TestAgent. This release includes the most powerful 7B test‑domain model and a framework for rapid local model deployment and experience.

Project address

https://github.com/codefuse-ai/Test-Agent

Local Mac M1 experience

MoDa experience

Experience address: https://modelscope.cn/studios/codefuse-ai/TestGPT-7B-demo/summary

Current Features

Model : Open‑source TestGPT‑7B, based on CodeLlama‑7B and fine‑tuned for downstream testing tasks.

Multi‑language test case generation (Java/Python/JavaScript) – improves readability, scenario coverage, and language support compared with traditional tools; future versions will add Go, C++.

Test case Assert completion – automatically adds missing asserts to existing test cases, boosting quality.

Engineering framework :

ChatBot page

Model quick start

Private deployment for secure, data‑leak‑free interaction

Future iterations will add more test‑domain applications such as knowledge Q&A, scenario analysis, and expand the model family to 13B and 34B.

Performance of the 7B Test Domain Model

The default TestGPT‑7B model outperforms existing open‑source models in pass@1 and average test‑scenario coverage.

Multi‑language test case generation

Pass@1 results for Java, Python, and JavaScript are shown below.

Test case Assert completion

Current support for Java assert completion with the following pass@1 result.

Architecture

The testing domain LLM leverages extensive world knowledge from pre‑training to exhibit strong reasoning and decision‑making in complex interactive environments.

While foundational models achieve impressive results, specialized testing tasks still require domain‑specific tools and knowledge. Integrating dedicated tools with the base model combines the strengths of both: tools address timeliness, expertise, interpretability, and robustness, while the model provides human‑like reasoning.

Quick Start

Prerequisites

Model download

Obtain model details and files from ModelScope or HuggingFace:

https://modelscope.cn/models/codefuse-ai/TestGPT-7B

https://huggingface.co/codefuse-ai/TestGPT-7B

Environment installation

git clone https://github.com/codefuse-ai/Test-Agent
cd Test-Agent
pip install -r requirements.txt

Ensure the execution environment has about 14 GB of GPU memory before running TestGPT‑7B.

Start services

Start the controller: python3 -m chat.server.controller Start the model worker (example for Mac M1):

python3 -m chat.server.model_worker --model-path models/testgpt --device mps

Device options: --device mps – GPU acceleration on Apple Silicon or AMD GPUs. --device xpu – Intel XPU acceleration (requires Intel Extension for PyTorch and OneAPI env). --device npu – Huawei AI processor acceleration (requires Ascend PyTorch Adapter and CANN env). --device cpu – CPU‑only execution. --num-gpus – Specify number of concurrent GPUs.

Start the web service: python3 -m chat.server.gradio_testgpt After the service is ready, open the local web address (http://0.0.0.0:7860) to see the full UI.