Separating Middleware Logs with Programmatic Log4j2 Configuration in Java

Background : The architecture team maintains dozens of middleware components (RPC, MQ, monitoring, task scheduling, etc.) that all log through SLF4J, mixing middleware logs with business logs and causing noise in troubleshooting and unnecessary storage consumption in Hadoop.

Solution Overview : Introduce a dedicated logging configuration inside LoggerContext, package it in a common jar ( zzarch-log-common), expose an API to obtain the dedicated logger, and let middleware use this logger for low‑importance logs such as status, heartbeat, and MQ events.

2.1 Adding Dedicated Log Configuration : The core class ArchLogConfigure programmatically creates a rolling file appender, a logger named LOGGER4ARCH, and registers them with Log4j2. The relevant Java code is:

public class ArchLogConfigure {
    private static final String SERVICE_NAME = ApplicationUtil.getApplicationName();
    public static final String FILE_NAME_FORMAT = "/opt/log/arch/%s/arch.log";
    public static final String FILE_PATTERN_FORMAT = "/opt/log/arch/%s/arch.log.%%d{yyyy-MM-dd-HH}";
    public static final String LOG_PATTERN = "%d{HH:mm:ss.SSS} %X{TRACE_ID}-%X{SPAN_ID}-%X{SAMPLED} %t %p %C{1.}:%L - %m%n";
    public static final String ARCH_APPENDER_NAME = "ARCH_APPENDER";
    public static final String ARCH_LOGGER_NAME = "LOGGER4ARCH";
    private static volatile boolean inited = false;

    /** Initialize */
    public static void init() {
        if (inited) { return; }
        synchronized (ArchLogConfigure.class) {
            if (inited) { return; }
            init0();
        }
    }

    private static void init0() {
        try {
            final LoggerContext loggerContext = (LoggerContext) LogManager.getContext(false);
            final Configuration configuration = loggerContext.getConfiguration();
            // 1. Create arch appender
            Appender archAppender = initRollingRandomAccessFileAppender(loggerContext);
            AppenderRef archAppenderRef = AppenderRef.createAppenderRef(ARCH_APPENDER_NAME, null, null);
            // 2. Create logger
            AppenderRef[] refs = new AppenderRef[] { archAppenderRef };
            LoggerConfig loggerConfig = AsyncLoggerConfig.createLogger("false", "INFO", ARCH_LOGGER_NAME, "true", refs, null, configuration, null);
            loggerConfig.addAppender(archAppender, null, null);
            // 3. Register logger and update context
            configuration.addLogger(ARCH_LOGGER_NAME, loggerConfig);
            loggerContext.updateLoggers();
            inited = true;
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    private static Appender initRollingRandomAccessFileAppender(LoggerContext loggerContext) {
        TriggeringPolicy policy = TimeBasedTriggeringPolicy.createPolicy("1", null);
        PatternLayout layout = PatternLayout.newBuilder().withPattern(LOG_PATTERN).build();
        String fileName = String.format(FILE_NAME_FORMAT, SERVICE_NAME);
        String fileNamePattern = String.format(FILE_PATTERN_FORMAT, SERVICE_NAME);
        RollingRandomAccessFileAppender appender = RollingRandomAccessFileAppender.createAppender(
            fileName, fileNamePattern, null, ARCH_APPENDER_NAME, "true", null, policy, null, layout, null, null, null, null, null, loggerContext.getConfiguration());
        appender.start();
        loggerContext.getConfiguration().addAppender(appender);
        return appender;
    }
}

The above class is equivalent to adding the following XML configuration to log4j2.xml:

<RollingRandomAccessFile name="ARCH_APPENDER" fileName="/opt/log/arch/${SERVICE_NAME}/arch.log" filePattern="/opt/log/arch/${SERVICE_NAME}/arch.log.%%d{yyyy-MM-dd-HH}">
    <PatternLayout pattern="%d{HH:mm:ss.SSS} %X{TRACE_ID}-%X{SPAN_ID}-%X{SAMPLED} %t %p %C{1.}:%L - %m%n" />
    <Policies>
        <TimeBasedTriggeringPolicy interval="1" modulate="true" />
    </Policies>
</RollingRandomAccessFile>
<AsyncLogger name="LOGGER4ARCH" level="INFO" includeLocation="true" additivity="false">
    <AppenderRef ref="ARCH_APPENDER" />
</AsyncLogger>

2.2 Exposing the Logger API : A simple factory class provides access to the dedicated logger.

public class ArchLoggerFactory {
    static { ArchLogConfigure.init(); }
    public static Logger getArchLogger() {
        return LoggerFactory.getLogger("LOGGER4ARCH");
    }
}

Middleware components can now obtain the logger via ArchLoggerFactory.getArchLogger() and use it exactly like any SLF4J logger.

3. Usage Example :

public class ArchLoggerFactoryTest {
    private final Logger archLogger = ArchLoggerFactory.getArchLogger();
    private final Logger logger = LoggerFactory.getLogger(this.getClass());

    @Test
    public void test() throws InterruptedException {
        for (int i = 0; i < 1000; i++) {
            archLogger.debug("name={}", "adu");
            archLogger.info("name={}", "adu");
            archLogger.warn("name={}", "adu");
            archLogger.error("name={}", "adu");

            logger.debug("name={}", "adu");
            logger.info("name={}", "adu");
            logger.warn("name={}", "adu");
            logger.error("name={}", "adu");

            Thread.sleep(1000);
        }
    }
}

Conclusion : By programmatically adding a dedicated logger configuration to LoggerContext and exposing a simple factory method, middleware can log low‑importance information to separate files without affecting business log analysis, thereby avoiding noise during troubleshooting and reducing big‑data storage costs.

The approach is straightforward yet yields significant benefits: it prevents middleware logs from interfering with business issue diagnosis and cuts down unnecessary storage expenses.

Author : Du Yunjie, architect, head of the architecture department at ZhaiZhai, member of the technical committee, responsible for service governance, MQ, cloud platform, APM, distributed tracing, monitoring, configuration center, task scheduling, ID generation, distributed locks, etc. Contact via WeChat ID waterystone.