Can Python Handle Millions of Requests per Second? Inside Japronto’s High‑Performance Microframework

Recent advances in Python performance—such as CPython 3.6’s new dictionary implementation, faster call rules in CPython 3.7, and PyPy’s JIT compiler—show that Python can still compete in high‑throughput scenarios, especially for web and micro‑service development.

Japronto is a brand‑new microframework designed for micro‑services, aiming for speed, scalability, and lightweight operation. In many tests it outperforms NodeJS and Go, thanks to asyncio which allows mixing synchronous and asynchronous code.

Benchmark comparisons on an AWS EC2 c4.2xlarge instance (8 vCPUs, Ubuntu 16.04.1, Python 3.6 compiled from source) using wrk (1 thread, 100 connections, 24 requests per connection) show that Japronto reaches 1,214,440 requests per second (RPS). The tests also reveal that Go’s stdlib HTTP server can be about 12 % faster than the illustrated Go implementation, while the fasthttp library is only ~18 % slower than Japronto.

HTTP pipelining is a key factor: by processing a batch of small GET requests in a single TCP packet, the server reduces system‑call overhead. Japronto parses the request stream, executes the requests as quickly as possible, merges the responses in the correct order, and performs a single system call to send the data back. Although scatter/gather I/O could further reduce copying, Japronto currently does not use it.

Additional performance techniques include:

Most core components (parser, protocol handling, connection management, routing, request/response objects) are written in C extensions.

Lazy loading of protocol headers and other objects, creating them only when first accessed.

Use of the ultra‑fast picohttpparser library, which leverages SSE4.2 instructions for rapid HTTP token matching.

Integration with uvloop, a libuv‑based event loop that uses epoll for asynchronous I/O.

Memory management is tuned to minimize garbage‑collector pressure: Japronto pre‑allocates memory in 4 KB blocks, reuses objects across requests, and avoids unnecessary reference cycles and allocations. It also reduces cache‑misses by keeping data in contiguous memory regions and avoids redundant copying during routing.

Current feature set includes full HTTP 1.x support with chunked uploads, complete HTTP pipelining, configurable keep‑alive, synchronous and asynchronous view handling, master‑multiworker processing, hot code reloading, and a simple routing syntax.

Future work will focus on WebSocket support and asynchronous response streaming.