Why High‑QPS Systems Move from JSON to Efficient Binary Serialization

What Serialization Actually Does

Serialization converts in‑memory objects—including types, references, and nested structures—into a byte stream that can be transmitted over the network or stored on disk; deserialization performs the reverse. Each RPC round‑trip requires at least two serializations and two deserializations.

Why JSON Was the Default Choice

JSON became popular not for speed but for convenience: browsers support it natively, curl can display it as readable text, and virtually every language provides a mature library. It requires no pre‑defined schema, so adding or changing fields is trivial, enabling rapid iteration when services are first split and traffic is low (hundreds of thousands of QPS).

The Hidden Costs of JSON

Three cost categories emerge:

Payload size : numbers like 12345 occupy five characters instead of a two‑byte integer, and field names (e.g., userId, createTime) are repeated in every record, often consuming half the message size for a ten‑field object.

CPU for parsing : Text parsing must scan characters, locate quotes and colons, and convert strings to numbers; the reverse happens during serialization, adding substantial CPU when executed millions of times per second.

GC pressure : JSON parsing creates many temporary string objects, increasing garbage‑collection pauses in JVM‑based runtimes.

When the Cost Becomes Painful

At million‑QPS levels the serialization overhead is a background noise, but at ten‑million‑QPS it becomes a dominant cost because the fixed per‑call expense is multiplied by the massive call volume and deep micro‑service call chains. Each additional internal call adds another pair of (de)serializations, amplifying the total cost beyond a simple ten‑fold increase.

First Evolution Step: Binary Schemas (Protobuf, Thrift)

Switching to a schema‑based binary format reduces payload size and CPU:

Field names are replaced by numeric identifiers.

Numbers are stored in their native binary representation.

Varint encoding lets small numbers occupy fewer bytes.

An illustration (see image) shows the same object encoded as JSON versus Protobuf, with the latter shrinking to a fraction of the original size and decoding several times faster.

The trade‑off is the need to maintain .proto or .thrift files, generate language‑specific code, and adhere to compatibility rules (e.g., never reuse field numbers). Debugging also requires specialized tools rather than raw text inspection.

No One‑Size‑Fits‑All Serialization

A comparison table (image) highlights that the optimal choice depends on four axes: schema maintenance cost, payload size, encoding speed, and readability. JSON remains best for public APIs where readability matters; Protobuf/Thrift dominate internal high‑frequency RPC; MessagePack offers a middle ground when schema maintenance is undesirable.

Second Evolution Step: Extreme Performance (FlatBuffers, Cap’n Proto, Compression)

When serialization becomes a CPU bottleneck, teams may adopt zero‑copy formats like FlatBuffers or Cap’n Proto, which lay out data so fields can be read directly from the byte buffer without materializing objects, benefiting read‑heavy, latency‑sensitive paths.

Alternatively, adding compression on top of binary data can save bandwidth, but only when messages are large enough to offset the extra CPU for (de)compression; applying compression to tiny RPC payloads typically degrades performance.

Compatibility Is the Hidden Constraint

Large systems with hundreds of services must support forward and backward compatibility. Schema‑based formats such as Protobuf encode field numbers, allowing new fields to be added without breaking older services, which simply ignore unknown numbers. JSON lacks enforced compatibility, relying on developers to keep interpretations consistent, which becomes risky at scale.

From Flexibility to Efficiency

The overall trajectory is clear: as scale grows, the design focus shifts from "flexibility first" to "efficiency first." At low QPS, developers pay for readability and rapid iteration; at tens of millions of QPS, every byte and CPU cycle matters, forcing a move to binary, zero‑copy, and selective compression.

Practical Guidance

Do not replace one format with another indiscriminately. A common pattern is to keep JSON for external APIs, use Protobuf for internal high‑throughput RPC, and reserve zero‑copy or compression for the few latency‑critical core paths where the performance gain outweighs added complexity.

Finally, examine your own services: is there a core chain where serialization already consumes a large CPU share? Are there low‑frequency calls where an over‑engineered zero‑copy solution adds maintenance burden without benefit? Balancing these factors yields the optimal serialization strategy for your system.