How I Boosted a Python Service to 50k QPS: Real‑World Performance Tuning Guide

"The purpose of this article is to record a Python program performance optimization, the problems encountered, and how they were solved, offering an optimization mindset; note that my approach is not the only one, and many solutions exist for performance challenges."

How to Optimize

Optimization must be driven by clear goals and real performance bottlenecks; vague concurrency numbers are meaningless without context.

Requirement Description

The module was split from the main site to handle high traffic, with targets: QPS ≥ 30,000, DB load ≤ 50%, server load ≤ 70%, request latency ≤ 70 ms, error rate ≤ 5%.

Environment configuration:

Server: 4‑core, 8 GB RAM, CentOS 7, SSD Database: MySQL 5.7, max connections 800 Cache: Redis, 1 GB

All services are hosted on Tencent Cloud.

Load testing tool: Locust with Tencent elastic scaling for distributed testing.

Key Analysis

The main tasks are: 1) fetch appropriate popup configuration for users, 2) record the next time to return configuration, 3) log user actions on the configuration.

Optimization

Database reads and writes for these tasks become a bottleneck under high load. To reduce DB pressure, write operations are off‑loaded to a FIFO queue implemented with a Redis list, and all configurations are cached, reading from the DB only on cache miss.

Initial load test after queuing writes showed QPS around 6,000, 502 errors rising to 30%, CPU 60‑70%, and DB connections saturated (≈800). The issue was identified as excessive DB reads for configuration lookup.

After caching all configurations, a second test reached ~20,000 QPS, CPU 60‑80%, DB connections ~300, and TCP connections ~15,000 per second.

However, TCP connections did not scale proportionally, suggesting socket limits. Checking ulimit -n showed 65,535, so the limit was raised to 100,001, but QPS only increased to ~22,000.

The root cause was TCP sockets lingering in TIME‑WAIT state, preventing reuse. Linux does not expose a direct parameter to shorten TIME‑WAIT, but the net.ipv4.tcp_max_tw_buckets setting controls the maximum number of TIME‑WAIT sockets.

# timewait count, default 180000
net.ipv4.tcp_max_tw_buckets = 6000
net.ipv4.ip_local_port_range = 1024 65000
# enable fast recycle of TIME‑WAIT sockets
net.ipv4.tcp_tw_recycle = 1
# allow reuse of TIME‑WAIT sockets for new connections
net.ipv4.tcp_tw_reuse = 1

After applying these kernel tweaks, the final load test achieved 50,000 QPS, CPU around 70%, stable DB connections, normal TCP connections, average response time 60 ms, and 0% error rate.

Conclusion

The optimization process highlighted the importance of understanding the interplay between web code, databases, networking, and operating system settings. Effective performance tuning requires solid fundamentals across these domains.

Note: Enabling tcp_tw_recycle and tcp_tw_reuse can cause issues in some environments; trade‑offs were made for this case. See related articles for deeper TCP insights.