Load Testing and Performance Analysis of the Online Quality Inspection Report Service

The online front‑end Node.js server for quality‑inspection reports experienced CPU saturation, causing errors when users accessed the report; a load‑testing project was initiated to determine the maximum concurrent load and to verify system reliability.

Objectives : (1) Identify performance bottlenecks and optimize them; (2) Simulate high‑probability single‑point failures to validate reliability; (3) Measure performance across various access scenarios to understand the system’s capacity.

Test Environment : A diagram of the architecture is provided, and the testing tools include Apache JMeter (version 4.0) for generating load.

Test Plan :

Test tools: JMeter.

Target interfaces: Front‑end API (https://*.com/youpin/next/qcreport) and back‑end API (https://*.com/business/api/ypqc/report).

Scenarios: three load patterns – 100, 500, and 1000 concurrent users, increasing by 10 or 20 users per second, each sustained for 10–20 minutes, monitoring response time, TPS, and successful transactions.

Support Personnel : Responsibilities for Nginx, MySQL, Redis, and other services were assigned, and they were briefed on the test plan to ensure rapid recovery if services fail during testing.

Test Data : 100,000 real orders were exported from production to avoid cache hits.

Distributed JMeter Setup :

Four sandbox machines were provisioned with JMeter. The following configuration steps were performed:

# Edit jmeter.properties

server.rmi.ssl.disable=true

server.rmi.localport=1099

server_port=1099

# Start JMeter server

./jmeter-server

Agents were added to remote_hosts (e.g., 10.9.193.122:1099,10.9.193.87:1099,...) and the test plan (.jmx) and data files were copied to each agent. The master launched the test with “Run‑Remote Start All”.

Server Monitoring : Spotlight was used for real‑time monitoring; SSH keys and port forwarding were configured (modifying /etc/ssh/sshd_config, restarting sshd, generating RSA keys, copying public keys, setting up a SOCKS5 proxy via Proxifier, and using CRT for port forwarding) to enable password‑less access and tunnel traffic.

Test Results and Analysis :

Front‑end test (scenario 1, 100 concurrent users, 10 min): CPU reached 100 %, throughput dropped, and response time increased, indicating a front‑end bottleneck.

Back‑end test (same load): Throughput grew with virtual users while response time remained stable as server CPU usage stayed low.

Problem Diagnosis :

Connection count spiked to >6000; suspected lack of connection reuse. Implemented Keep‑Alive on both front‑end and back‑end to reduce connections.

Analyzed Nginx logs; identified long‑latency interfaces. Coordinated with back‑end team to improve those APIs and reduced timeout from 5 s to 1 s.

Conclusion : After applying Keep‑Alive and timeout optimizations, a subsequent load test showed no CPU saturation and all metrics remained normal; the system’s capacity and reliability were validated.