Optimizing Log4j2 Logging: Reducing Disk and CPU Usage, Asynchronous Logging, and Traceability with ThreadLocal and TTL

As the shopping‑cart service of JD.com experiences growing traffic, excessive logging creates high disk pressure, CPU load, and difficulty in tracing request chains. The article outlines three main improvement areas.

1. Reduce Resource Usage

1.1 Lower Disk Usage

By classifying logs and printing only INFO level during development and WARN level in production, daily disk usage stays around 10 GB. Log level hierarchy (ALL < TRACE < DEBUG < INFO < WARN < ERROR < FATAL < OFF) is used to filter logs.

Dynamic log‑level adjustment code (shown in the image) allows switching levels without redeployment.

1.2 Asynchronous Logging

Switching from synchronous logging to Log4j2 AsyncLogger dramatically reduces latency; synchronous INFO logging raises request time from 320 ms to 500 ms, while asynchronous logging adds negligible overhead.

AsyncLogger offers two modes: global async (set via JVM parameter log4j2.contextSelector=AsyncLoggerContextSelector ) and mixed async, allowing selective async for performance‑critical logs.

AsyncLogger internally uses the LMAX Disruptor ring buffer, providing lock‑free, high‑throughput log handling.

1.3 Lower CPU Usage

Printing location information (class, file, line) increases CPU consumption up to 30‑100×. By disabling location data in PatternLayout and using log level filtering, CPU usage is kept low even under 20 GB/h write rates.

2. Log Traceability

Standard Log4j2 NDC/MDC only works within a single thread. To propagate trace IDs across thread pools, the system combines TransmittableThreadLocal (TTL) with Log4j2 MDC.

2.1 ThreadLocal Technology Selection

ThreadLocal – isolates variables per thread (no child‑thread support).

InheritableThreadLocal – copies values to newly created child threads (fails with thread pools).

TransmittableThreadLocal – copies values to tasks submitted to thread pools, requiring thread‑pool modification.

2.2 TTL Thread‑Pool Refactoring

Thread‑pool code is rewritten to capture and restore TTL context before and after task execution, ensuring trace IDs flow correctly.

2.3 Log4j2 MDC + TTL Implementation

MDC stores contextual data; with TTL, a snapshot of the main thread’s MDC is copied to worker threads, enabling full request‑level logging across asynchronous processing.

2.4 Trace Log Usage

Each request is assigned a traceId that propagates through the entire business flow, allowing precise problem isolation.

3. Test Metrics and Recommendations

3.1 Recommendations

Record essential runtime parameters for quick issue diagnosis.

Use log level control to reduce CPU and disk pressure.

Design clear log formats to avoid performance risks.

Combine Log4j2 with monitoring tools for faster problem detection.

Employ asynchronous logging when performance is critical; apply log level filtering for high‑volume logs.

Refactor thread pools to simplify context propagation.

3.2 Test Indicators

CPU usage tests on a 4‑core, 8 GB server show significant reductions when disabling location info and using async logging.

3.3 Log4j2 Parameter Test Results

Various Log4j2 configuration parameters (e.g., %C, %F, %L) were benchmarked; enabling these dramatically degrades performance, confirming the need for minimal pattern layouts in high‑throughput environments.

Images illustrating the metrics and configurations are retained from the original article.