From HTTP to Private RPC: Why Million‑QPS Systems Need Protocol‑Level Optimization

Why Everyone Started with HTTP

HTTP was a zero‑cost choice in the early stages of a service because it was universally understood, easy to test with browsers, curl, or Postman, and its plain‑text format made debugging straightforward. When QPS was only in the tens of thousands, the development speed and debugging convenience outweighed the protocol overhead.

What Makes HTTP Expensive at High Frequency

At millions of QPS, the following costs dominate:

Textual redundancy: HTTP headers such as Content-Type: application/json, User-Agent: ..., and other header lines add hundreds of bytes to a request whose payload may be only a few dozen bytes.

CPU cost of parsing text headers: each request requires scanning the header line‑by‑line, splitting strings, and converting them into structured data, which becomes a significant CPU burden at high call rates.

JSON serialization overhead: converting numbers to strings and repeatedly transmitting field names inflates payload size and adds CPU‑intensive type inference during deserialization.

These costs are proportionally small at 100k QPS but become the main contributor to latency and resource consumption at ten‑million QPS.

Step 1: Switch to Binary Serialization

Replacing JSON with binary formats such as Protobuf or Thrift reduces payload size dramatically and speeds up (de)serialization by several times. The binary schema maps field names to numeric identifiers, cutting the transmitted bytes to a fraction of the original size.

Benefits include lower CPU usage and smaller network traffic, while trade‑offs involve schema maintenance, code generation for multiple languages, and the loss of human‑readable payloads.

Step 2: Adopt a Private Protocol Instead of HTTP

Even after binary serialization, the surrounding HTTP envelope still adds unnecessary bytes and parsing work. A custom private protocol can replace the HTTP header with a compact binary header (typically a few dozen bytes) containing a magic number, version, message type, body length, and request ID.

The fixed‑size header enables O(1) parsing without string scanning, dramatically reducing CPU cost compared to O(n) text parsing. However, this sacrifices universality and easy debugging, requiring dedicated tooling for observability.

Step 3: Move from Short Connections to Long‑Lived Connection Reuse

Traditional HTTP often uses a new TCP connection per request, incurring three‑way handshakes and four‑way teardowns. At high QPS, this handshake overhead becomes a major waste, especially when TLS is involved.

Private protocols naturally pair with persistent connections and connection pools, amortizing handshake costs. Multiplexing multiple requests over a single connection requires a request ID in the protocol header to match responses, and adds complexity such as keep‑alive handling and pool capacity management.

When Not to Use a Private Protocol

Private protocols are justified only for high‑frequency internal communication where protocol overhead is a bottleneck. External APIs must remain HTTP for compatibility, and low‑frequency internal calls (e.g., admin tasks) do not benefit enough to offset the added maintenance burden.

Observability is another hidden cost: the rich ecosystem of HTTP tracing, logging, and gateway metrics must be re‑implemented for a private protocol, often requiring a full RPC framework that bundles serialization, discovery, load balancing, and tracing.

Evolution Essence: From General‑Purpose to Performance‑First

The progression from HTTP + JSON to HTTP + binary, then to private protocol + long‑connection reuse illustrates a shift in design focus: as scale grows, the priority moves from universal compatibility to raw performance.

At ten‑million QPS, every redundant byte and every extra handshake is multiplied by massive call volumes, turning previously negligible overhead into a critical bottleneck. The solution is not to discard HTTP entirely but to layer specialized, high‑performance protocols where they matter most, while keeping HTTP for external or low‑frequency paths.