Comparative Performance Evaluation of JMeter, Locust, asyncio (Python) and Go for Server Load Testing

Server-side performance testing is a major demand in the testing field. Established tools such as JMeter and LoadRunner, developed abroad, remain the primary solutions due to their excellent design and implementation.

In recent years, alternative solutions have emerged, notably Python and Go, which have grown rapidly; Python benefits from fast and simple development cycles, while Go offers high performance and easy concurrent programming.

Common Solutions

JMeter – https://github.com/apache/jmeter

LoadRunner – http://www8.hp.com/us/en/software-solutions/loadrunner-load-testing/

Locust (Python 2.7) – https://github.com/locustio/locust

asyncio (Python 3.5) – https://github.com/python/asyncio

Go – https://github.com/golang/go

JMeter is not described in detail.

LoadRunner is a commercial product and therefore not included in the comparison.

Locust is a recent single‑threaded, multi‑coroutine load‑testing framework built on Python 2.7 greenlet/gevent.

asyncio is the built‑in coroutine library of Python 3.5, optimized compared to 2.7 and provides related HTTP request libraries such as https://github.com/KeepSafe/aiohttp .

Go’s built‑in support for concurrency, combined with its C‑like performance, makes it a primary language for high‑performance server components; its HTTP library is available at https://github.com/valyala/fasthttp .

Testing Objectives

From a performance‑testing engineer’s perspective, the goal is to achieve higher concurrency with fewer hardware resources, selecting the most efficient solution (low resource consumption, high pressure) as the implementation plan.

Test Environment

Server: CentOS, 4 CPU × 16 GB RAM

Server software: openresty 2 workers (similar to Nginx)

Client: Ubuntu, 2 CPU × 2 GB RAM

GET request for a static page

Test Plan

Measure client CPU and memory consumption, server CPU load, and key TPS data for concurrent users of 10, 50, 100, 500, and 1000.

HTTP GET requests target an Nginx static page; disk I/O load is low and not considered in the comparison.

Test Scripts

Refer to the repository https://github.com/likezjuisee/blog for the test scripts; dependencies and tool usage should follow the official documentation, which is not detailed here.

Locust requires distributed execution using the two CPUs of the client server.

Go and asyncio scripts print the total transaction count every 3 seconds, which can be converted to TPS.

Comparison Results

Locust

Concurrency

CPU

Memory

Nginx

TPS

10

197

2.8

11

770

50

199

3.0

11

760

100

199

3.5

11

730

500

200

5.6

9

680

1000

200

8.1

9

680

asyncio

Concurrency

CPU

Memory

Nginx

TPS

10

199

2.2

16

1300

50

199

2.2

18

1300

100

199

2.4

17

1400

500

200

3.0

15

1500

1000

200

3.8

15

1200

JMeter

Concurrency

CPU

Memory

Nginx

TPS

10

189

13.5

60

8700

50

196

14.4

60

8200

100

196

19.0

60

7750

500

200

32.0

60

7100

1000

200

32.0

60

6700

Go

Concurrency

CPU

Memory

Nginx

TPS

10

129

0.5

70

11000

50

193

0.6

125

19000

100

194

0.6

120

18000

500

195

1.0

120

16000

1000

196

1.3

120

16000

Conclusion

Python 3.5 coroutine implementation improves performance by about 100% over 2.7 at 1000 concurrent users; a performance‑testing framework can be built on aiohttp.

JMeter’s Java‑thread implementation outperforms Python, but compiled languages still have an edge; Python frameworks are more extensible for multiple protocols, while the latest JMeter does not support WebSocket.

JMeter’s TPS drops noticeably under high concurrency, likely due to its multithreaded approach; Go’s TPS remains stable.

JMeter’s multithreaded model consumes significantly more memory compared to coroutine‑based solutions; Go performs best in memory usage.

In TPS, Go delivers the highest results—more than twice that of JMeter—especially in high‑concurrency scenarios.

Vision

A Go‑based performance testing platform is under development; stay tuned.