Why Log4j2 Async Logging Blocks Threads and How to Fix It

Logging is a critical infrastructure in software engineering for monitoring, diagnosing, and tracing system behavior. Apache Log4j2, a widely used logging framework, offers modular and extensible features, but improper use of its asynchronous logging and buffer strategies can cause I/O blocking, excessive memory consumption, and severe performance bottlenecks.

1. Current Logging Situation

Log blocking due to queue saturation.

Choosing logging policies from development to production.

2. Causes of the Problem

2.1 Logging Principle

In our application we use Log4j2’s async configuration. A log event passes through filters and wrappers, then is placed into a Disruptor ring buffer. A dedicated consumer thread dequeues events and writes them to the target file.

2.2 Disruptor Initialization

When LoggerContext starts, each AsyncLoggerConfig calls start() to initialize its Disruptor. The default ring‑buffer size is 256 KB.

2.3 Queue Full Leads to Blocking

If the ring buffer is full, the default behavior AsyncLoggerConfig.SynchronizeEnqueueWhenQueueFull=true makes the producer wait for a free slot, acquiring a global lock and blocking all logging threads.

2.4 Root Cause

Producer speed exceeds consumer speed. The consumer thread may be slowed by slow appenders (e.g., disk I/O) or frequent flush calls, causing the buffer to accumulate and eventually block producer threads.

3. Mitigation Strategies

3.1 Solution Selection

Address the issue from both producer and consumer sides. Adjust queue capacity cautiously (watch for OOM), and redesign log policies.

3.2 Technical Choices

Classify logs into functional logs (essential for observability) and diagnostic logs (optional, enable on demand).

Implement line‑level log control to dynamically enable/disable specific log statements without restarting.

3.3 Implementation Details

Four approaches are discussed:

Custom Appender Filter : Parse stack traces to filter logs by line number. Drawbacks include unreliable line detection in lambdas, performance overhead, and poor integration with existing log levels.

Embed Line Info in Log Message : Manually add class and line info, which is not scalable.

Compile‑time Bytecode Injection : Use a Maven plugin to modify bytecode after compilation, inserting class/line metadata without runtime cost. ASM is chosen for bytecode manipulation.

IDE Plugin for Runtime Control : An IntelliJ IDEA plugin reports the enable/disable state of specific log lines to a configuration service (e.g., Apollo), allowing on‑the‑fly toggling without blocking the main thread.

3.4 Maven Plugin & Idea Plugin Workflow

The Maven plugin runs in the process‑classes phase, modifies compiled classes to embed log location data, and incurs no runtime overhead. The IDEA plugin captures user actions to toggle log lines, reports the state to Apollo, and the custom logging utility reads this configuration to decide whether to emit the log.

4. Summary

In modern distributed systems, log management has become a cornerstone of observability. This article reveals how Log4j2 async logging can block threads when the Disruptor queue fills, and proposes a dual‑track log management model that separates essential business logs from optional diagnostic logs. By leveraging compile‑time bytecode enhancement and IDE‑integrated control, developers gain surgical‑level log management, avoiding the blunt‑force approach of traditional log level filtering while maintaining system stability under high load.