Introduction to Protocol Buffers (ProtoBuf) with Java Example and Performance Comparison

ProtoBuf is a Google‑developed tool for efficiently storing and reading structured data. Structured data refers to records that share a common schema, such as a phone‑book entry containing name, ID, email, and phone number.

Compared with XML and JSON, ProtoBuf produces a more compact binary representation, resulting in smaller payloads and faster processing.

The ProtoBuf compiler (protoc) translates language‑agnostic .proto definition files into source code for specific languages (Java, C/C++, Python, etc.). The generated classes can be used via the language‑specific runtime library.

Installation (Mac): brew install protobuf

Example workflow (Java) :

1. Create a message.proto file

syntax = "proto3";

message Person {
    int32 id = 1;
    string name = 2;

    repeated Phone phone = 4;

    enum PhoneType {
        MOBILE = 0;
        HOME = 1;
        WORK = 2;
    }

    message Phone {
        string number = 1;
        PhoneType type = 2;
    }
}

2. Place the .proto file under src/main/proto in a Java project.

3. Compile the proto file to Java classes

From the src/main directory run:

protoc --java_out=./java ./proto/*.proto

The generated Java classes appear in src/main/java .

4. Add the ProtoBuf Java runtime dependency (Gradle example)

implementation 'com.google.protobuf:protobuf-java:3.9.1'

5. Serialize a Java object to ProtoBuf

Message.Person.Phone.Builder phoneBuilder = Message.Person.Phone.newBuilder();
Message.Person.Phone phone1 = phoneBuilder
        .setNumber("100860")
        .setType(Message.Person.PhoneType.HOME)
        .build();
Message.Person.Phone phone2 = phoneBuilder
        .setNumber("100100")
        .setType(Message.Person.PhoneType.MOBILE)
        .build();
Message.Person.Builder personBuilder = Message.Person.newBuilder();
personBuilder.setId(1994);
personBuilder.setName("XIAOLEI");
personBuilder.addPhone(phone1);
personBuilder.addPhone(phone2);
Message.Person person = personBuilder.build();
long start = System.currentTimeMillis();
byte[] buff = person.toByteArray();
System.out.println("ProtoBuf encoding time: " + (System.currentTimeMillis() - start));
System.out.println("ProtoBuf data length: " + buff.length);

6. Deserialize ProtoBuf data back to a Java object

System.out.println("-Start decoding-");
long start = System.currentTimeMillis();
Message.Person personOut = Message.Person.parseFrom(buff);
System.out.println("ProtoBuf decoding time: " + (System.currentTimeMillis() - start));
System.out.printf("Id:%d, Name:%s\n", personOut.getId(), personOut.getName());
for (Message.Person.Phone phone : personOut.getPhoneList()) {
    System.out.printf("Phone:%s (%s)\n", phone.getNumber(), phone.getType());
}

Performance comparison (encoding/decoding time and data size) between JSON (using GSON) and ProtoBuf over multiple runs:

JSON encoding 1 time: 22ms, length 106 bytes
ProtoBuf encoding 1 time: 32ms, length 34 bytes
... (similar results for 10, 100, 1000, 10000, 100000 runs) ...
ProtoBuf consistently shows smaller payload (≈1/3 of JSON) and faster decode/encode when the number of operations exceeds a few thousand.

Summary of findings :

For fewer than 1,000 operations, ProtoBuf performance is comparable to JSON, sometimes slightly slower.

Beyond 2,000 operations, ProtoBuf outperforms JSON in both encoding and decoding.

At 100,000+ operations, ProtoBuf’s advantage becomes very pronounced.

Memory usage: ProtoBuf data occupies 34 bytes versus 106 bytes for JSON (about one‑third).

Compatibility – adding or removing fields:

Adding a new field (e.g., nickname ) to the .proto file and regenerating Java classes allows old binary data to be parsed successfully, with the new field defaulting to empty.

Id:1994, Name:XIAOLEI
Phone:100860 (HOME)
Phone:100100 (MOBILE)
getNickname=

Removing an existing field (e.g., name ) also works; the missing field is read as null without breaking deserialization.

Id:1994, Name:null
Phone:100860 (HOME)
Phone:100100 (MOBILE)

Overall, ProtoBuf provides efficient serialization, smaller payloads, and forward/backward compatibility, making it a strong alternative to JSON for data exchange in Java applications.