Using SLF4J MDC to Correlate Logs Across Threads and Asynchronous Tasks in Java

When a request fails, we often need to collect all logs generated during that request to locate the problem.

If the request is processed in a single thread, filtering by thread ID works, but in asynchronous processing the thread ID is insufficient.

Huawei IoT platform receives device data and processes it asynchronously (storage, rule engine, push, command, etc.). Because the processing steps are loosely coupled, they run in separate threads, making log correlation challenging. This article shows how to quickly filter all logs belonging to a single request.

1. Basic usage

The SLF4J logging framework provides the MDC (Mapped Diagnostic Context) utility, which can store a request identifier in the logging context.

public class Main {
    private static final String KEY = "requestId";
    private static final Logger logger = LoggerFactory.getLogger(Main.class);

    public static void main(String[] args) {
        // entry: put request ID
        MDC.put(KEY, UUID.randomUUID().toString());

        // log some messages
        logger.debug("log in main thread 1");
        logger.debug("log in main thread 2");
        logger.debug("log in main thread 3");

        // exit: remove request ID
        MDC.remove(KEY);
    }
}

After configuring log4j2.xml , running the program prints logs that contain the requestId inside curly braces, e.g.:

2018-02-17 13:19:52.606 {requestId=f97ea0fb-2a43-40f4-a3e8-711f776857d0} [main] DEBUG cn.wudashan.Main - log in main thread 1
2018-02-17 13:19:52.609 {requestId=f97ea0fb-2a43-40f4-a3e8-711f776857d0} [main] DEBUG cn.wudashan.Main - log in main thread 2
2018-02-17 13:19:52.609 {requestId=f97ea0fb-2a43-40f4-a3e8-711f776857d0} [main] DEBUG cn.wudashan.Main - log in main thread 3

Because the requestId appears in every log line, we can filter logs for a specific request with a simple grep requestId=xxx *.log command.

2. Advanced scenario – asynchronous threads

When a new thread is created, the MDC value does not propagate, so the requestId is missing in the async thread’s logs:

public class Main {
    private static final String KEY = "requestId";
    private static final Logger logger = LoggerFactory.getLogger(Main.class);

    public static void main(String[] args) {
        MDC.put(KEY, UUID.randomUUID().toString());

        logger.debug("log in main thread");

        new Thread(new Runnable() {
            @Override
            public void run() {
                logger.debug("log in other thread");
            }
        }).start();

        MDC.remove(KEY);
    }
}

The missing requestId occurs because MDC is implemented with ThreadLocal , which stores data only in the current thread.

To solve this, we can use the decorator pattern and create an MDCRunnable that captures the current MDC map and restores it in the target thread.

public class MDCRunnable implements Runnable {
    private final Runnable runnable;
    private final Map
map;

    public MDCRunnable(Runnable runnable) {
        this.runnable = runnable;
        // Save current thread's MDC values
        this.map = MDC.getCopyOfContextMap();
    }

    @Override
    public void run() {
        // Restore MDC values
        for (Map.Entry
entry : map.entrySet()) {
            MDC.put(entry.getKey(), entry.getValue());
        }
        runnable.run();
        // Clean up
        for (Map.Entry
entry : map.entrySet()) {
            MDC.remove(entry.getKey());
        }
    }
}

We then wrap any asynchronous task with MDCRunnable :

public class Main {
    private static final String KEY = "requestId";
    private static final Logger logger = LoggerFactory.getLogger(Main.class);
    private static final ExecutorService EXECUTOR = Executors.newSingleThreadExecutor();

    public static void main(String[] args) {
        MDC.put(KEY, UUID.randomUUID().toString());

        logger.debug("log in main thread");

        new Thread(new MDCRunnable(new Runnable() {
            @Override
            public void run() {
                logger.debug("log in other thread");
            }
        })).start();

        EXECUTOR.execute(new MDCRunnable(new Runnable() {
            @Override
            public void run() {
                logger.debug("log in other thread pool");
            }
        }));
        EXECUTOR.shutdown();

        MDC.remove(KEY);
    }
}

Running this program produces logs where the requestId is present in the main thread, the explicit thread, and the thread‑pool thread:

2018-03-04 23:44:05.343 {requestId=5ee2a117-e090-41d8-977b-cef5dea09d34} [main] DEBUG cn.wudashan.Main - log in main thread
2018-03-04 23:44:05.346 {requestId=5ee2a117-e090-41d8-977b-cef5dea09d34} [Thread-1] DEBUG cn.wudashan.Main - log in other thread
2018-03-04 23:44:05.347 {requestId=5ee2a117-e090-41d8-977b-cef5dea09d34} [pool-2-thread-1] DEBUG cn.wudashan.Main - log in other thread pool

Congratulations! The requestId now appears in all asynchronous logs.

3. Summary

This article demonstrated how to use SLF4J’s MDC to tag logs with a request identifier and how to extend its functionality to asynchronous threads using the decorator pattern. By injecting the requestId via AOP or similar mechanisms, any workflow’s logs can be filtered quickly, greatly speeding up debugging and operational analysis.

Using MDC during development reduces the time needed to locate issues, and in production it enables rapid collection of relevant logs for faster incident response.