Analysis of DBCP Connection‑Pool Bug Causing Database Connection Saturation and Recommended Mitigations

Experiment background – Over the past year several incidents of database connections being exhausted were observed; initial analysis pointed to the application using more connections than the DBCP 1.4 pool limit, possibly due to a pool bug.

Problem analysis – The DBCP bug can create multiple pool instances during module initialization, causing the pool size to exceed its configured maximum. The root cause is the protected synchronized DataSource createDataSource() method that may be invoked repeatedly when createConnectionPool() fails.

Key code fragment (createConnectionPool)

protected void createConnectionPool() {
    // Create an object pool to contain our active connections
    GenericObjectPool gop;
    if (abandonedConfig != null && abandonedConfig.getRemoveAbandoned()) {
        gop = new AbandonedObjectPool(null, abandonedConfig);
    } else {
        gop = new GenericObjectPool();
    }
    gop.setMaxActive(maxActive);
    gop.setMaxIdle(maxIdle);
    gop.setMinIdle(minIdle);
    gop.setMaxWait(maxWait);
    gop.setTestOnBorrow(testOnBorrow);
    gop.setTestOnReturn(testOnReturn);
    gop.setTimeBetweenEvictionRunsMillis(timeBetweenEvictionRunsMillis);
    gop.setNumTestsPerEvictionRun(numTestsPerEvictionRun);
    gop.setMinEvictableIdleTimeMillis(minEvictableIdleTimeMillis);
    gop.setTestWhileIdle(testWhileIdle);
    connectionPool = gop;
}

Configuration parameters examined – The article lists a typical DBCP configuration (driverClassName, URL, maxActive = 25, maxIdle = 5, maxWait = 3000 ms, logAbandoned = true, removeAbandoned = true, removeAbandonedTimeout = 18 s, etc.) and highlights the importance of removeAbandonedTimeout when connections are long‑running.

Test setup – A Docker MySQL 5.7 container is started, a test database is created, and a JUnit test creates a BasicDataSource with maxActive=1 , removeAbandoned=true , and removeAbandonedTimeout=1 . Twenty threads each execute a slow query SELECT SLEEP(100), NOW() while the pool attempts to reclaim abandoned connections.

public void testAbandoned() throws Exception {
    for (int i = 0; i < 20; i++) {
        Thread t = new Thread(() -> {
            try (Connection conn = ds.getConnection();
                 Statement stmt = conn.createStatement();
                 ResultSet rs = stmt.executeQuery("select sleep(100),now()")) {
                while (rs.next()) {
                    System.out.println("result" + rs.getString(1));
                }
            } catch (Exception e) {
                e.printStackTrace();
            }
        });
        t.setName(String.valueOf(i));
        t.start();
        Thread.sleep((i + 1) * 1000);
    }
    Thread.sleep(1000000);
}

Observations – When socketTimeout=1000 is set, the removeAbandoned logic triggers, the pool closes the connection while the SQL is still executing, and MySQL logs “The last packet successfully received … 1,001 ms ago”. Without removeAbandoned the problem does not appear, but many threads wait for the timeout.

Conclusion – Enabling removeAbandoned=true together with a short removeAbandonedTimeout can cause the pool to discard connections that are still processing queries, leading to a mismatch between pool size and actual MySQL connections. The issue is reproduced only with MySQL‑connector‑java < 5.1.45.

Optimization recommendations

Disable removeAbandoned for mixed AP/TP workloads where long‑running queries are common.

Remove the socketTimeout parameter or upgrade to a newer MySQL driver where the bug is fixed.

Ensure the connector version is ≥ 5.1.45 to avoid the bug.