How to Generate an API Test Plan Using Prompt Engineering

1. Introduction

In API testing, a test plan defines goals, scope, and procedures. Traditionally it requires significant manual effort, but using carefully designed prompts to generate a test plan can save time while ensuring completeness.

2. Preparation

2.1 Gather API Information

Collect all relevant details of the target API: purpose, HTTP method (GET, POST, PUT, DELETE, etc.), request parameters (name, type, required), response format (e.g., JSON), and expected results. An example is a user‑login API that uses POST, with username (string, required) and password (string, required) and returns {"success": true, "userInfo": {...}} on success or an error object on failure.

2.2 Define Core Test‑Plan Content

Identify the mandatory sections for any test plan: test objectives, scope, environment, test‑case design, execution steps, defect management, and risk assessment. Clarify objectives (e.g., verify functionality, performance, security), scope (which endpoints to test), and environment details (servers, databases, tools).

3. Prompt Design Principles

3.1 State the Requirement Clearly

The prompt must explicitly ask the AI to generate an API test plan and embed the collected API information and the core sections identified above. Vague prompts like “generate a test plan” lead to generic outputs.

3.2 List Key Information as Bullet Points

Present API details in a structured list to help the model parse them. Example format:

Interface name: User login API

Function: Validate username and password

Method: POST

Parameters:

username – string, required, 6‑20 alphanumeric characters

password – string, required, 8‑20 characters with special symbols

Response format: JSON

Success result: {"success":true,"code":200,"message":"登录成功","data":{...}}

Failure results: various error objects (e.g., username empty, password empty, user not found)

3.3 Specify Output Format

Require the plan to be rendered in Markdown with hierarchical headings (H1 for main sections, H2 for subsections) and lists for test cases. Example instruction: “Use Markdown, H1 for modules such as Test Objectives, H2 for sub‑items, and list each test case with ID, scenario, parameters, expected result, and priority.”

4. Write the Initial Prompt

4.1 Prompt Structure

The initial prompt should contain three parts: requirement description, detailed API information, and test‑plan output requirements.

4.2 Example Prompt

For the login API:

Requirement: Generate a detailed API test plan for the following interface.

API Details:
- Name: User login API
- URL: http://www.example.com/api/login
- Function: Validate credentials and return user info and token; return error messages for invalid input.
- Method: POST
- Headers: Content-Type: application/json
- Parameters:
  * username: string, required, 6‑20 alphanumeric characters
  * password: string, required, 8‑20 characters with special symbols (!@#$%^&*)
- Response format: JSON
- Success example:
  {
    "success": true,
    "code": 200,
    "message": "登录成功",
    "data": {
      "userId": "123456",
      "username": "testuser",
      "token": "eyJhbGciOi..."
    }
  }
- Failure examples:
  {"success": false, "code": 400, "message": "用户名不能为空"}
  {"success": false, "code": 401, "message": "密码错误"}

Test‑plan output requirements:
- Include objectives, scope, environment, strategy, test‑case design, execution steps, defect management, risk assessment.
- Test cases must contain ID, scenario, pre‑conditions, steps, parameters, expected result, priority.
- Output in Markdown with proper headings and ordered lists.

5. Generate the Initial Test Plan

Paste the prompt into an AI tool (e.g., ChatGPT, Doubao). Review the generated plan for completeness and adherence to the required sections.

5.1 Choose an AI Tool

Different tools may produce varying detail and style. Test a few prompts with each tool and select the one that best matches project needs.

5.2 Submit Prompt and Generate

Enter the prompt, trigger generation, and wait for the full plan to be produced without interrupting the model.

5.3 Preliminary Review

Check that all core sections are present, that API details are correctly reflected, and that the Markdown format follows the specification. Note any missing items for later refinement.

6. Refine Prompt and Test Plan

6.1 Analyse Issues

Common problems include vague objectives, incomplete test cases, and insufficient environment description.

6.2 Optimize Prompt

Amend the prompt to add missing details, such as more precise objectives (e.g., “verify login success, handle empty username, invalid password, performance under 100 concurrent users, and SQL‑injection resistance”), or to request additional test scenarios.

6.3 Regenerate and Re‑check

Run the updated prompt, then verify that the new plan addresses the previously identified gaps.

7. Fine‑Tune the Test Plan

7.1 Detail Test‑Case Steps

For each case, add explicit execution steps (e.g., open Postman, create POST request to the URL, set headers, input parameters, send request, compare response).

7.2 Add Test Data

Provide concrete data for each scenario, such as a 4‑character username for “username too short” or a password without special symbols for “missing special character”.

7.3 Define Schedule

Specify review, design, execution, and reporting dates (e.g., “Plan review: 2023‑05‑01 14:00‑15:00”).

7.4 Assign Roles

List responsibilities for each tester (e.g., Tester A handles normal login and empty‑username cases, Tester B handles password‑related cases, Tester C handles length‑validation and missing‑header cases).

8. Review and Adjust

8.1 Identify Reviewers

Include test lead, testers, developers of the API, and product manager.

8.2 Conduct Review Meeting

Distribute the plan beforehand, then discuss each section, record minutes, and note required changes.

8.3 Apply Review Feedback

Update the plan to add missing scenarios (e.g., request‑parameter type errors) and ensure consistent formatting.

8.4 Finalize

After approval, circulate the final plan to all stakeholders.

9. Execute and Validate

9.1 Execute Tests

Follow the plan precisely, log each step, record timestamps, parameters, actual results, and compare with expectations.

9.2 Submit and Track Defects

Log any failures in a defect‑tracking tool (e.g., Jira) with severity, priority, and screenshots.

9.3 Verify Effectiveness

Assess whether performance targets (e.g., ≤1 s response time under 100 concurrent users, ≥99.9 % success rate) were met, whether coverage includes all defined scenarios, and whether defect discovery rate meets expectations.

9.4 Record Summary

Document total cases, pass/fail counts, uncovered defects, lessons learned, and suggestions for future plans.

10. Optimize Prompt Template for Future Projects

10.1 Summarize Strengths and Weaknesses

Strengths: detailed API info and explicit test‑case fields lead to accurate plans. Weaknesses: missing formatting details for schedules, insufficient test‑data specifications.

10.2 Refine Template

Update the template to require schedule tables, explicit test‑data sources, and security‑testing items.

10.3 Reuse and Adjust

For new APIs, replace the API‑specific sections while keeping the overall structure.

11. Common Issues and Solutions

11.1 Mismatch Between Generated Plan and Real Needs

Ensure the prompt mentions all required aspects (e.g., security testing) and provides complete error‑response examples.

11.2 Duplicate or Missing Test Cases

State in the prompt that scenarios must be unique and cover all key functions, including edge cases like pagination errors.

11.3 Incorrect Formatting

Specify exact Markdown heading levels, table syntax, and field order in the prompt.

11.4 Overly Concise Content

Demand detailed execution steps, environment preparation, and mitigation measures for each identified risk.

By iteratively refining prompts and reviewing AI‑generated output, teams can efficiently produce high‑quality, executable API test plans.