Why Pingora Is Overtaking Nginx as the New Web Server Champion

Background

Cloudflare processes >1 trillion client requests per day. Its existing Nginx‑based edge proxy reached limits in performance, scalability, and feature extensibility.

Limitations of Nginx

Single‑worker process per request leads to CPU core load imbalance.

Connection pools are scoped to a worker; adding workers fragments pools, reducing connection reuse.

Adding complex features (e.g., per‑retry header sets) requires invasive changes to upstream C code.

C implementation lacks memory‑safety guarantees, increasing bug and security risk.

Evaluated Options

Continue investing in Nginx and pay for customizations – feasible but would demand massive engineering effort.

Migrate to a third‑party proxy such as Envoy – risked repeating similar limitations.

Build an internal platform from scratch – highest upfront cost but promised best long‑term fit.

Design Decisions for Pingora

Language: Rust was chosen for memory safety without sacrificing performance.

HTTP library: A bespoke library was written instead of reusing hyper, to maximise flexibility and allow rapid innovation.

Non‑RFC‑compliant traffic: The library tolerates many edge‑case HTTP behaviours observed on the open internet (e.g., status codes 599‑999).

Workload scheduling: Multithreaded model with work‑stealing (Tokio runtime) replaces the multiprocess model, enabling a shared connection pool.

Programmable request lifecycle: An event‑driven interface similar to Nginx/OpenResty lets developers inject custom logic (request filters, header manipulation) without modifying core code.

Production Performance Gains

Median TTFB reduced by 5 ms; 95th‑percentile TTFB reduced by 80 ms.

Connection reuse rate increased from 87.1 % to 99.92 %, cutting new connections by a factor of ~160.

CPU usage dropped ~70 % and memory usage ~67 % for the same traffic load.

New connections per second fell to one‑third of the previous service.

Efficiency Mechanisms

Shared‑thread connection pools eliminate per‑worker fragmentation, reducing TLS handshakes.

Rust’s zero‑cost abstractions avoid the Lua‑on‑NGINX copy‑and‑GC overhead when accessing HTTP headers.

Atomic reference‑counted shared objects replace mutex‑protected shared memory used by Nginx.

Extensibility

HTTP/2 upstream support added with minimal effort, enabling future gRPC services.

Cache Reserve integrates Cloudflare R2 as a storage layer for caching.

Programmable interface allows developers familiar with Nginx/OpenResty to transition quickly.

Safety

Rust’s memory‑safety semantics eliminated crashes caused by undefined behaviour. After processing millions of billions of requests, no service‑code‑induced crashes were observed; rare crashes were traced to kernel or hardware faults.

Conclusion

Pingora provides a faster, more efficient, and more flexible internal proxy platform for Cloudflare. The project is planned to be open‑sourced after further maturation.

Code example

最近面试BAT，整理一份面试资料
《
Java面试BAT通关手册
》
，覆盖了Java核心技术、JVM、Java并发、SSM、微服务、数据库、数据结构等等。
获取方式：点“
在看
”，关注公众号并回复
手册
领取，更多内容陆续奉上。
明天见(｡･ω･｡)