Why Cloudflare Replaced Nginx with Pingora: Inside the New High‑Performance Proxy

Background

Cloudflare processes more than a trillion client requests per day. The existing Nginx‑based edge proxy exhibited several architectural limits that prevented further scaling and feature development.

Limitations of the Nginx Worker Model

Each request is bound to a single worker process, causing load imbalance across CPU cores.

Connection pools are per‑worker, so increasing the number of workers reduces connection‑reuse efficiency.

Adding advanced per‑request logic (e.g., custom header manipulation, retry policies) required invasive changes to Nginx’s C code.

Nginx’s C implementation lacks memory‑safety guarantees, increasing the risk of crashes and security bugs.

Evaluation of Alternatives

Continue investing in Nginx and pay for customizations.

Migrate to a third‑party proxy such as Envoy.

Build an internal platform from scratch.

After multiple years of assessment, the team concluded that a custom solution offered the best long‑term return on investment.

Pingora Project

Design Decisions

Language: Implemented in Rust to achieve C‑level performance while providing memory‑safety.

HTTP Stack: A bespoke HTTP library was created instead of using existing crates (e.g., hyper) to handle the wide range of non‑RFC‑compliant traffic seen on the Internet.

Concurrency Model: Multithreaded execution on the Tokio async runtime with work‑stealing. This enables shared resources such as connection pools across threads, eliminating the per‑worker isolation of Nginx.

Programmable Request Lifecycle: An API modeled after Nginx/OpenResty allows developers to insert custom logic (request filters, header modifications, retry handling) without touching core proxy code.

Production Performance

Median time‑to‑first‑byte (TTFB) reduced by 5 ms; 95th‑percentile TTFB reduced by 80 ms.

Connection‑reuse rate increased from 87.1 % to 99.92 % for a major client, cutting new connections by a factor of 160 and saving an estimated 434 years of TLS handshake time per day.

CPU usage dropped ~70 % and memory usage ~67 % under comparable load, thanks to shared connection pools and more efficient Rust code compared with Lua on Nginx.

Additional Capabilities

HTTP/2 upstream support was added quickly, paving the way for gRPC services.

Cache Reserve feature uses Cloudflare R2 storage as an external caching layer.

Safety and Reliability

Rust’s memory‑safety guarantees eliminate many classes of crashes and security vulnerabilities common in C‑based code. Since its launch, Pingora has processed trillions of requests without a service‑code‑induced crash. The multithreaded model also reduces synchronization overhead compared with Nginx’s shared‑memory locks.

Conclusion

Pingora is now the primary internal HTTP proxy at Cloudflare, delivering lower latency, higher connection‑reuse, and substantially reduced CPU and memory consumption. The project is planned to be open‑sourced after further maturation.