How to Shrink Log Files: Cut 5GB Daily Logs to Under 1GB with Proven Techniques

In daily development, many INFO‑level logs are printed for debugging, but as traffic grows a log file can exceed size thresholds (e.g., 5 GB per day), leading to performance issues during log retrieval, data collection, and disk cleanup.

Log Slimming Methodology

Print only necessary logs

Sometimes developers temporarily add many INFO logs for testing; before production these can be removed or downgraded to DEBUG. When a log could be either DEBUG or INFO, you can use a switch to temporarily promote DEBUG logs to INFO during troubleshooting.

Example pseudo‑code:

if(log.isDebugEnable()){
    log.debug(xxx);
}
else if(TracerUtils.openDebug2Info()){
    log.info("【debug2info】"+xxx);
}

Prefix such logs with a marker like 【debug2info】 to distinguish them from regular INFO logs.

Merge printing

Combine related log statements into a single entry. For example, two INFO logs before and after a method can be merged:

INFO [64 traceId] XXXService 执行前 size =10 INFO [64 traceId] XXXService 执行后 size =4

can become:

INFO [64 traceId] XXXService 执行前 size =10 执行后 size =4

Simplify & abbreviate & compress

Print only the ID of large objects.

Create a lightweight log‑specific DTO that retains only key fields.

Use abbreviations (e.g., w for write, r for read) or short identifiers (e.g., S1, S2) for frequently logged beans.

Optimization Case Study

Scenario description

A business flow involves many beans that inherit from a common abstract class, which logs the number of items before and after each bean execution.

Optimization analysis

1. Print only necessary logs

Since the pre‑execution log is essentially the previous bean’s post‑execution log, keep only the post‑execution INFO and downgrade the pre‑execution log to DEBUG.

If the size before and after execution is identical, skip logging.

Pseudo‑code:

if(sizeBefore != sizeAfter){
    log.info("service:{}, 前size:{},后size:{}", getName(), sizeBefore, sizeAfter);
}

This reduces redundant logs dramatically when most beans have unchanged sizes.

2. Log merging

Record the pre‑execution size in memory and log both before and after sizes in a single statement:

log.info("service:{}, 前size:{},后size:{}", getName(), sizeBefore, sizeAfter);

3. Log simplification

Convert result objects (e.g., XXDTO) to a lightweight log object containing only essential fields such as id and title before printing:

log.info("resultId:{}", result.getId());

or

log.info("result:{}", toSimpleLog(result));

Effect evaluation

The original logs generated about 5 GB per day, with roughly 80 % consisting of size‑before/after entries and 10 % of final results. After applying the above optimizations, daily log volume dropped to under 1 GB while still supporting issue diagnosis.

Conclusion

Log slimming requires balancing the need for diagnostic information with storage efficiency. Techniques include removing unnecessary logs, merging related entries, and simplifying log content. Additionally, you can temporarily promote DEBUG logs to INFO (or use tools like Arthas) to aid troubleshooting without inflating log size.