Memory Leak Analysis and Optimization of Protocol Buffers in a Core Backend Module

In a Baidu quality assurance project, a critical backend module exhibited continuous memory growth over months, eventually triggering out‑of‑memory alarms and frequent restarts. The issue manifested differently in production (steady increase) versus test environments (stabilization), complicating reproduction.

The team first ruled out data hot‑loading and performed a binary‑search style investigation across the module's 13 sub‑modules, narrowing the leak to a specific sub‑module (module A). Unit tests were introduced to accelerate verification, reducing each validation cycle to about 30 minutes.

Monitoring revealed that the merged_data structure inside module A continuously expanded. Further inspection showed that the Protocol Buffers clear() method only resets fields without releasing the allocated memory, causing the cache to grow unchecked when large messages are repeatedly cleared.

To resolve the problem, the clear operation was replaced with explicit deletion and reallocation using a scoped pointer reset, ensuring that memory is freed each cycle. Benchmark graphs (shown as images in the original article) demonstrate a significant reduction in memory growth and even a slight improvement in CPU idle time after the change.

The authors conclude that Protocol Buffers' clear cache mechanism is unsuitable for large, frequently changing messages; regular deletion or recreation of messages is recommended. They also emphasize the importance of fine‑grained memory and CPU monitoring, as well as the efficiency of targeted unit tests for diagnosing similar performance issues.