How to Build a 100% Test‑Coverage Automated Backend Testing System

Background and Pain Points

Backend developers often encounter recurring bugs, version‑compatibility issues, and heavy mental load when modifying legacy code. Existing tools such as WeJestAPITest helped but still suffered from skipped failures, slow case iteration, low coverage, and difficulty for developers to maintain test cases.

Failed cases are frequently marked as skipped, rendering automation ineffective.

Rapid requirement changes cause test cases to lag behind, increasing maintenance cost.

Developers find it hard to participate in case maintenance; testers write overly simple cases, leading to low coverage.

Why Build a Custom System

Open‑source tools (JMeter, Postman) and internal solutions (WeTest, ITEST) are ready‑to‑use but do not satisfy specific business needs such as custom scheduling, RPC framework support, seamless integration with deployment and alert systems, and a low learning curve for backend engineers.

Goals

Language‑agnostic, ideally no‑code, to eliminate barriers between different programming languages.

Separate test case authoring from the test execution service; updating cases should not require service changes.

Support scenario testing (multiple cases forming a flow) with variable references between cases.

Provide flexible scheduling: full‑batch, module‑level, case‑group, or single‑case dispatch.

Manage both HTTP and RPC cases, covering upstream and downstream call chains.

Minimize the effort for backend engineers to write test cases.

Overall Architecture

The system consists of a proxy layer, a unified request description, a parameter‑pool, validation components, and an asynchronous MQ scheduler. Test cases are stored separately and can be triggered by various sources (change system, management platform, routine tasks).

Unified HTTP and RPC Access

Both request types share a common description format:

HTTP: http://host:port/urlpath + reqbody RPC: rpc://ip:port/method + reqbody By configuring a module name and a client, the system automatically determines whether the request is HTTP or RPC and routes it accordingly.

Parameter Pool Construction

Complex service flows often require downstream requests to reuse values from upstream responses. The system extracts these values using regular expressions or JSONPointer paths and stores them as key‑value pairs in a parameter pool. A lightweight template engine then replaces placeholders in downstream requests with the corresponding values, turning inter‑case dependencies into on‑demand lookups.

JSON Schema Component

Instead of writing hand‑crafted assertions for each field, the system uses JSON Schema to describe the expected shape of response data. The schema can enforce required fields, data types, value ranges, and enumerations, providing high readability and language‑independent validation.

{
  "type": "object",
  "required": ["bookId", "title", "author", "cover", "format", "price"],
  "properties": {
    "bookId": {"type": "string", "const": "123456"},
    "title": {"type": "string", "minLength": 1},
    "author": {"type": "string", "minLength": 1},
    "cover": {"type": "string", "format": "uri"},
    "format": {"type": "string", "enum": ["epub", "txt", "pdf", "mobi"]},
    "price": {"type": "number", "exclusiveMinimum": 0}
  }
}

Advantages include high readability, a single source of truth for all rules, language‑agnostic usage, and the ability to generate schemas automatically from real JSON samples.

JSONPath Component

Some validation scenarios require relational checks between fields (e.g., updateTime >= createTime) or assertions on array contents. JSONPath provides expressive extraction and comparison capabilities.

// updateTime >= createTime
$.updateTime > $.createTime

// bookId must equal a fixed value
$.bookId == ["123456"]

// datas array must be non‑empty
$.datas.length > [0]

// datas must contain a book with price > 0
$.datas[?(@.bookId=="123456")] > [0]

JSON Schema and JSONPath can be used together or independently, depending on the test scenario.

Change System Integration and Scheduling

The platform uses an asynchronous MQ to dispatch test tasks. Scheduling supports multiple trigger sources, granularities (full batch, module, case group, single case), and environment selection (online, test, or specific IP).

Coverage Measurement

Coverage is measured at two levels:

Global coverage = Covered interfaces / Total interfaces * 100%

Effective test cases = All enumerable parameter values + all parameter combinations

Interface coverage = Covered effective cases / Effective test cases * 100%

When an interface reaches 100% coverage, it is considered fully tested.

Test Case Generation from Traffic

Instead of manually writing cases, the system analyzes online traffic to extract parameter features (count, type, value range, enumerability, combinability). Enumerable parameters are identified by a dynamic threshold, and their combinations are enumerated to generate test cases automatically. Each generated case includes a JSON Schema for response validation.

Case Discovery and Completion

Two offline tasks keep the suite up‑to‑date:

New interface discovery – when traffic shows a previously unseen request, it is added to the test pool.

New case discovery – differences between traffic‑derived parameters and existing cases trigger automatic case creation.

Results and Summary

After deployment, the number of test cases grew from ~150 to >8,000, achieving 100% read‑interface coverage. The system continuously evolves JSON Schemas from live traffic, provides early detection of abnormal data, and reduces the mental burden on developers by automating case generation, scheduling, and validation.

Key take‑aways are a generic, language‑agnostic testing framework architecture and a traffic‑driven methodology for measuring and improving test coverage.