Performance Optimization of a SpringMVC + Dubbo Based Offline Payment System

The project is an offline payment transaction system that uses SpringMVC as the web front‑end (deployed on Tomcat) and Dubbo for the backend services; SpringMVC calls Dubbo to complete business functions.

Optimization Goal : Without changing business logic, use stress testing to measure interface performance during order placement and, if problems appear, improve JVM settings, database connection pools, and Dubbo connection pools.

Preparation for Stress Testing

1. Testing Strategy : Incrementally increase concurrency, starting from 1 thread and raising it until a bottleneck appears, then adjust parameters.

2. Stress‑test Branch : Create a feature_stress branch from release to isolate configuration changes from other developers.

3. Test Environment : Allocate three 4C‑32G VMs—one for Tomcat, one for Dubbo services, and one for MySQL—to evaluate single‑machine performance.

4. Test Script : Use JMeter to simulate the offline payment flow.

Initial Stress Test : Starting with 1 concurrent request, the test quickly hit many connection failures and memory‑overflow errors; throughput was only 1 req/s, while Tomcat thread count peaked above 20 000.

Problem Analysis : A custom Log4j appender created a new thread pool for each log entry, causing an explosion of threads. The appender was configured in log4j.properties and added to rootLogger .

Initial Optimization : Refactor the appender so the thread pool is a class‑level field limited to 8 threads, then redeploy.

Result: Throughput jumped to 1 700 req/s.

Further Tuning

Even after the first fix, thread count remained high (~3 000). Investigation showed many threads were Dubbo consumer connections; the default consumer connection limit of 1000 was reduced to 64, and console logging in Log4j was disabled.

Final Result: Throughput reached 2 600 req/s.

Summary of Optimization Points

1. CPU was not a bottleneck because the project performed little computation.

2. IO and excessive threads/connections were the main bottlenecks; reducing thread creation and connection counts improved performance.

3. Disable console output in production; it is synchronous and costly.

4. When throughput is low, examine thread stacks to identify blocking points, whether in service calls or database access.

For more technical articles and resources, see the links provided at the end of the original post.