Why Spring Boot APIs Hang: Debugging a Redis Connection Pool Deadlock

Problem description: In an internal sandbox environment, all APIs became unresponsive for a week, causing the application to appear deadlocked.

Initial attempts to restart the application temporarily restored service, but the issue recurred more frequently, prompting deeper investigation.

Using top on the server showed normal CPU usage, so the focus shifted to JVM thread analysis. The command top -H -p 12798 identified the three most resource‑intensive threads, and jstack was used to examine stack traces. jstack 12798 | grep 12799的16进制 31ff The stack traces revealed many threads in a lock state without any business‑related code, suggesting a bottleneck in the Redis connection pool.

Further debugging with Alibaba’s Arthas showed numerous http-nio threads in a waiting state, confirming that API requests were blocked.

Investigation of the Redis pool code uncovered that pool.getResource() caused threads to wait indefinitely because the borrowMaxWaitMillis parameter was not configured (default -1 means wait forever). The relevant snippet:

public T getResource() {
    try {
        return internalPool.borrowObject();
    } catch (Exception e) {
        throw new JedisConnectionException("Could not get a resource from the pool", e);
    }
}

public T borrowObject(long borrowMaxWaitMillis) throws Exception {
    if (p == null) {
        if (borrowMaxWaitMillis < 0) {
            p = (PooledObject) this.idleObjects.takeFirst();
        } else {
            waitTime = System.currentTimeMillis();
            p = (PooledObject) this.idleObjects.pollFirst(borrowMaxWaitMillis, TimeUnit.MILLISECONDS);
            waitTime = System.currentTimeMillis() - waitTime;
        }
        if (p == null) {
            throw new NoSuchElementException("Timeout waiting for idle object");
        }
    }
    return p;
}

Because borrowMaxWaitMillis was unset, threads could block forever, leading to the observed API hangs.

The fix involved configuring the Redis pool with a reasonable timeout and ensuring connections are properly released. Example configuration:

JedisConnectionFactory jedisConnectionFactory = new JedisConnectionFactory();
JedisPoolConfig config = new JedisPoolConfig();
config.setMaxWaitMillis(2000); // 2 seconds
jedisConnectionFactory.setPoolConfig(config);
jedisConnectionFactory.afterPropertiesSet();

After redeploying with the new settings, the issue resurfaced, and log analysis showed RedisConnectionFailureException: Cannot get Jedis connection caused by exhausted pool resources.

Root cause: Using stringRedisTemplate.getConnectionFactory().getConnection() directly acquires a pool connection without returning it, leaving the connection in a non‑idle state. The correct approach is to execute Redis commands within a RedisCallback so the framework can manage the connection lifecycle:

stringRedisTemplate.execute(new RedisCallback<Cursor>() {
    @Override
    public Cursor doInRedis(RedisConnection connection) throws DataAccessException {
        return connection.scan(options);
    }
});

Alternatively, manually release the connection after use:

RedisConnectionUtils.releaseConnection(conn, factory);

Additional recommendations include avoiding the KEYS command in production and configuring the pool size and timeout appropriately to prevent silent deadlocks.

Conclusion: Spring’s StringRedisTemplate abstracts most Redis operations but does not support certain commands like SCAN or SETNX directly; developers must obtain a low‑level Jedis connection via a callback and ensure it is released, while also setting proper pool parameters to avoid indefinite waits.