Why Logback’s Default packageData Triggers Dubbo Thread Locks and How to Resolve It

Phenomenon Description

During an offline load test of the core Dubbo interface getActivityList using the in‑house gatling‑dubbo framework, the mean response time (MRT) stayed around 41 ms at ~100 RPS but jumped to over 1 s when RPS approached 200, indicating a severe performance bottleneck.

Problem Analysis

Thread‑dump analysis with jstack showed threads blocked on an AladdinClassLoader lock and concurrently logging errors via Logback. Using Arthas confirmed that the same lock blocked 195 other threads. The GUI tool jmc identified the top‑level blocking object as the same AladdinClassLoader.

Further inspection of the application logs revealed frequent error stack traces that included the originating JAR and version information. This extra information is added by Logback’s PackagingDataCalculator.loadClass method, which forces class loading during each error log.

Running the command sc (the JDK jcmd shortcut) confirmed that the culprit JAR was logback‑classic‑1.1.2.jar. In this version, the packageData option is enabled by default, causing Logback to load class metadata for every exception stack trace.

Root Cause

When many threads simultaneously log exceptions, each thread attempts to acquire the exclusive AladdinClassLoader lock to load class metadata, leading to massive lock contention and the observed latency spike.

Solution

Two approaches were considered:

Temporarily avoid error logging by fixing the configuration that triggered the logs.

Upgrade Logback to version 1.2.3, where packageData is disabled by default.

After upgrading, the same interface handled over 500 RPS with an MRT of only 55 ms, eliminating the lock contention.

Extended Summary

Logging, while essential, can become a performance hazard if misconfigured. Common pitfalls include:

Improper logging configuration (e.g., enabling line‑number or package data output).

Excessive logging frequency, especially in high‑traffic paths.

Using outdated logging framework versions that introduce heavy lock contention.

Understanding the logging framework’s behavior and keeping it up‑to‑date are crucial for maintaining system stability under load.