Log Shrinking Techniques and Case Study for Reducing Log Size

Background

In daily development, developers often print many INFO‑level logs for debugging, which can cause log files to grow beyond thresholds (e.g., 5 GB per day). Oversized logs increase the cost of operations such as log retrieval, data collection, and disk cleanup.

Log Shrinking Methodology

1. Print Only Necessary Logs

Remove or downgrade non‑essential INFO logs to DEBUG before production. When a log may be needed for troubleshooting, a switch can temporarily promote DEBUG logs to INFO.

if (log.isDebugEnable()) {
    log.debug(xxx);
} else if (TracerUtils.openDebug2Info()) {
    log.info("【debug2info】" + xxx);
}

Prefix such promoted logs with a marker like 【debug2info】 to distinguish them.

2. Merge Duplicate Logs

Combine consecutive logs that share the same context into a single entry.

INFO [traceId] XXXService 执行前 size =10 INFO [traceId] XXXService 执行后 size =4

becomes

INFO [traceId] XXXService 执行前 size =10 执行后 size =4

3. Simplify, Abbreviate, Compress

When a log contains large objects, consider logging only the ID, creating a lightweight log DTO, or using abbreviations (e.g., w for write , r for read ).

log.info("resultId:{}", result.getId());

or

log.info("result:{}", toSimpleLog(result));

Optimization Case Study

Scenario Description

A business pipeline consists of many beans that inherit from a common abstract class, which logs the item count before and after each bean execution. The final bean prints the result.

Optimization Analysis

1) Print Only Necessary Logs : Keep only the post‑execution log; pre‑execution INFO logs are either removed or downgraded to DEBUG. Add a conditional check to skip logging when sizeBefore == sizeAfter :

if (sizeBefore != sizeAfter) {
    log.info("service:{}, 前size:{},后size:{}", getName(), sizeBefore, sizeAfter);
}

2) Merge Logs : Record sizeBefore in memory and output a single combined log after execution:

log.info("service:{}, 前size:{},后size:{}", getName(), sizeBefore, sizeAfter);

3) Log Simplification : Convert the result object to a lightweight DTO that contains only key fields (e.g., id , title ) before logging.

log.info("resultId:{}", result.getId());

Effect Evaluation

The original logs generated about 5 GB per day, with ~80 % coming from size‑before/after entries. After applying the three techniques, daily log volume dropped to less than 1 GB, achieving an 80 % reduction while still satisfying troubleshooting needs.

Conclusion

Log shrinking requires balancing the need for diagnostic information with storage efficiency. By deleting unnecessary logs, merging related entries, and simplifying log content, developers can significantly reduce log size without compromising the ability to troubleshoot issues.