How to Build a 100% Coverage Automated Backend Testing System

Background and Motivation

Backend developers often face recurring bugs, version‑compatibility headaches, and fragile test suites that either get skipped or cause heavy maintenance burdens. Existing tools such as WeJestAPITest, JMeter, and Postman provide basic functionality but fail to meet internal requirements for scalability, extensibility, and low‑cost maintenance.

Goals of the Self‑Built System

Language‑agnostic, ideally no‑code, to eliminate framework barriers.

Separate test cases from the test service so that case changes do not require service redeployment.

Support scenario testing with variable references across cases.

Provide flexible scheduling (full‑run, module‑level, case‑group, single‑case) and environment targeting.

Manage both HTTP and RPC interfaces uniformly.

Minimize the effort required for backend engineers to write cases.

Overall Architecture

The system consists of a proxy layer that routes requests to either logical (HTTP) or service (RPC) modules based on a configurable module name. Test cases are stored in a central repository, and an asynchronous MQ schedules execution.

Unified HTTP and RPC Access

Both request types are expressed in a unified syntax:

HTTP: http://host:port/urlpath + reqbody

RPC: rpc://ip:port/method + reqbody

This allows a single configuration (module name + client) to resolve the correct endpoint and protocol.

Parameter‑Pool Construction

Complex request chains often require downstream cases to reuse values from upstream responses. Instead of hard‑coding a linear pass‑through, a global parameter pool stores key‑value pairs where the value can be a literal or a JSONPointer path extracted from a previous response.

JSON Schema Component

To replace manual field‑by‑field assertions, the system generates JSON Schema rules for each response. Example code from the legacy framework shows the verbosity of hand‑written checks:

function bookInfoBaseCases(bookInfoObject) {
    it('expected bookInfo.bookId non‑empty string equals 12345', () => {
        expect(bookInfo.bookId).not.toBeNull();
        expect(typeof bookInfo.bookId).toEqual('string');
        expect(bookInfo.bookId).toEqual('12345');
    });
}

Problems include low scalability, JavaScript‑only syntax, and scattered assertions. The generated JSON Schema for the same payload is concise and language‑agnostic:

{
  "type": "object",
  "required": ["bookId","title","author","cover","format","price"],
  "properties": {
    "bookId": {"type": "string", "const": "123456"},
    "title": {"type": "string", "minLength": 1},
    "author": {"type": "string", "minLength": 1},
    "cover": {"type": "string", "format": "uri"},
    "format": {"type": "string", "enum": ["epub","txt","pdf","mobi"]},
    "price": {"type": "number", "exclusiveMinimum": 0}
  }
}

Advantages: high readability, single‑source rule management, no extra learning curve, and easy generation from real traffic.

JSONPath Component

JSON Schema cannot express relational checks (e.g., updateTime >= createTime) or cross‑object comparisons. JSONPath provides extraction and assertion capabilities:

// assert updateTime > createTime
$.updateTime > $.createTime

// assert fixed bookId
$.bookId == ["123456"]

// ensure array non‑empty
$.datas.length > [0]

// complex condition on array items
$.datas[?(@.bookId=="123456")] > [0]

Both JSON Schema and JSONPath can be combined for comprehensive validation.

Change‑System Integration and Scheduling

Test execution is triggered asynchronously via MQ with three key features:

Multiple trigger sources (change system, management platform, cron).

Granular scheduling (full run, module, case group, single case).

Environment selection (production, test, specific IP).

Coverage Measurement and Flow

Because protocol documentation is fragmented, the team derives coverage metrics from live traffic. The process extracts parameters from URL, headers, and JSON bodies, builds a feature set (count, type, range, enumerability, combinability), and defines a threshold (e.g., 30) to decide which parameters are enumerable.

Example traffic sample:

listType=[1,2,3,4]
listMode=[1,2]
vid=[10000,10001,...]
platform=[android,ios,web]
appver=[7.2.0,7.1.0,7.3.0]
/bookId=[12345,23456,...]
/filterType=[1,2]
/filterTags=[abc,def,efg]
/commOptions/opts1=[testops1,testops1_]
/commOptions/opts2=[testops2]

Enumerability analysis yields nine meaningful parameter combinations, which become the target for test‑case generation.

Coverage formulas:

GlobalCoverage = CoveredInterfaces / TotalInterfaces * 100%
EffectiveTestCases = AllEnumerableParameterValues + AllEnumerableParameterCombinations
InterfaceCoverage = CoveredEffectiveTestCases / EffectiveTestCases * 100%

Automated Test‑Case Generation

Using the extracted traffic features, the system automatically creates test cases and corresponding JSON Schema validators. The workflow includes:

Generate initial schema with genson (multi‑input capable).

Manually fine‑tune schema for business‑specific rules (e.g., minItems for result arrays).

Execute cases; on failure, run a promotion pipeline that either removes obsolete cases, replaces them with fresh ones, or adjusts the schema.

Case Discovery and Completion

Two offline jobs continuously discover:

New interfaces appearing in traffic.

New parameter enumerations or missing combinations, prompting automatic case creation.

Results and Impact

After migration, the platform grew from ~150 manual cases to >8,000 generated cases, achieving 100% read‑API coverage and dramatically reducing manual effort. The unified schema approach eliminated the need to maintain thousands of lines of test code, while the coverage metric guided continuous improvement.

Key achievements include:

Language‑agnostic JSON Schema + JSONPath validation.

Scenario testing via parameter pool and variable references.

Customizable scheduling integrated with deployment pipelines.

Seamless HTTP/RPC handling, lowering RPC test entry cost.

Automated case generation that boosted coverage from 150+ to 8,000+ cases.

Conclusion

The article presents a generic, extensible testing framework architecture and a traffic‑driven coverage measurement method that can be adapted to various backend services, demonstrating that full‑coverage automated testing is achievable with modest engineering investment.