How a Misimplemented Spring Bean Caused a Massive Memory Leak and How We Fixed It
The article details two incidents where a misconfigured Spring Bean caused thread blockage and Full GC in a Java backend, explains the step‑by‑step debugging using thread and GC dumps, and presents the code changes that finally eliminated the memory leak.
In this article we document two incidents of system errors caused by memory leaks in a Java Spring application, describe the monitoring observations, the step‑by‑step investigation, and the final fixes.
First incident – Full GC after code deployment
After deploying new code at 14:16, the service ran normally until memory usage and thread count began rising, leading to many blocked threads and frequent Full GC. Dump analysis showed thread HSFBizProcessor-DEFAULT-9-thread-792 blocked for 116 s, with 682 blocked threads. The blockage originated from the OssClient bean’s postProcessAfterInitialization method, where an OSS client was created. The OSS endpoint was misconfigured (South Tong data center accessed a Zhangbei OSS), causing network timeouts.
Fix: corrected the OSS configuration and restarted the machine, which stopped the blockage.
Second incident – Recurring Full GC
Three hours after a later deployment, Full GC reappeared, though with shorter duration. Dump analysis again pointed to the same OssClient code, but no blocked threads were found. GC dump revealed a massive number of org.apache.http.impl.conn.PoolingHttpClientConnectionManager instances (≈110 k), consuming over 80 % of heap memory. These objects are created indirectly by the OSS client and were never released because the client’s shutdown method was never called.
Root cause: the OssClient implemented BeanPostProcessor, causing the initialization logic to run on every bean creation, repeatedly creating OSS clients.
Final resolution
Changing the implementation to InitializingBean (or otherwise moving the initialization to a single place) ensures the OSS client is created only once and properly shut down, eliminating the memory leak.
These incidents illustrate why understanding Spring bean lifecycles and proper resource cleanup is essential for reliable backend services.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
Alibaba Cloud Developer
Alibaba's official tech channel, featuring all of its technology innovations.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
