Why Cloudflare Dropped Nginx for the New Pingora Proxy

Introduction

Cloudflare replaced its Nginx‑based edge proxy with a self‑built service called Pingora, written in Rust, to handle more than a trillion client requests per day.

Why Build a New Proxy

Architectural limits hurt performance

NGINX assigns each request to a single worker process. This creates load imbalance across CPU cores and locks a request to its worker, so heavy‑CPU or blocking‑IO work on one request degrades others.

Connection reuse suffers because each worker maintains an isolated connection pool. Adding workers spreads connections across pools, increasing hardware usage and response latency.

Some features are hard to add

Cloudflare needs advanced capabilities such as retrying failed requests to different upstreams with distinct header sets, which NGINX does not support natively.

Modifying NGINX’s C code is time‑consuming, risky, and the codebase lacks memory safety. Lua extensions reduce some risk but add performance overhead and lack static type checking. The NGINX community’s limited activity also hampers collaborative development.

Choosing to Build Our Own

Three options were evaluated over several years:

Continue investing in NGINX and pay for customizations, which would require massive engineering effort.

Migrate to another third‑party proxy such as Envoy, which risked repeating the same cycle of limitations.

Build an internal platform from scratch, requiring the largest upfront investment.

Quarterly reviews showed no clear formula for the best choice, but the projected return on investment of a self‑built proxy eventually appeared more compelling, leading to the decision to start from zero.

Pingora Project

Design decisions

Rust was chosen to achieve C‑level performance with memory safety.

A custom HTTP library was built instead of using existing libraries such as hyper, to maximize flexibility and to handle many non‑RFC‑compliant HTTP cases observed at Cloudflare’s scale.

Support for extended status codes (599‑999) was added; hyper initially rejected this change, prompting the custom implementation.

Workload scheduling uses a multithreaded model with the Tokio async runtime, enabling easy sharing of resources (e.g., connection pools) and work‑stealing to avoid the worker‑process bottlenecks of NGINX.

A programmable request‑lifecycle interface, similar to NGINX/OpenResty, allows developers to write filters that can modify or reject requests early, separating business logic from proxy logic.

Pingora is faster in production

Median time‑to‑first‑byte (TTFB) decreased by 5 ms and the 95th percentile decreased by 80 ms compared with the previous service.

Shared‑across‑thread connection pools improve reuse, reducing TCP/TLS handshake time.

New connections per second are one‑third of the old service; for a major client, connection reuse rose from 87.1 % to 99.92 %, a 160× reduction in new connections, saving an estimated 434 years of handshake time per day.

More Features

The developer‑friendly interface removes previous limitations, enabling rapid addition of core features such as new protocols.

Adding HTTP/2 upstream support (and consequently gRPC) required minimal effort in Pingora, whereas implementing the same in NGINX would have been far more labor‑intensive.

Pingora also integrates Cloudflare’s Cache Reserve, using R2 storage as a caching layer.

More efficient

Under identical traffic loads, Pingora consumes roughly 70 % less CPU and 67 % less memory than the legacy service.

Rust code runs faster than the previous Lua implementation and avoids costly string copying and garbage collection required by NGINX/OpenResty’s Lua bridge.

The multithreaded model enables efficient cross‑request data sharing; most shared objects are accessed via atomic reference counting instead of mutex‑protected shared memory.

Reduced new‑connection overhead further cuts TLS handshake costs.

More secure

Rust’s memory‑safety guarantees protect against undefined behavior, allowing higher development velocity at Cloudflare’s scale.

Since Pingora’s inception, millions of trillions of requests have been processed without a service‑code‑induced crash; rare crashes have been traced to hardware or kernel bugs, not the proxy itself.

Summary

Pingora provides a faster, more efficient, and more extensible internal proxy that underpins current and future Cloudflare product layers, with plans to open‑source it after further maturation.

Code example

资料链接
强烈建议大家使用 Linux 做开发?
豆包官宣收费！AI完全免费时代结束了
阿里二面：1000万条短信1小时发完，线程池怎么配？