Why Did Our API Hang for a Week? Uncovering Redis Connection Pool Deadlocks in Spring

In an internal sandbox environment the APIs stopped responding for a week, causing the whole application to appear frozen. Initial checks showed no errors in the IDE, and both the database and Redis were reported as normal.

SSH into the server and run top to verify system health, then inspect the JVM stack with jstack. The most resource‑intensive threads were identified, and the focus shifted to Redis connection handling.

Debugging revealed that threads were blocked while acquiring a Jedis connection from the pool. The relevant code in JedisConnectionFactory is:

protected Jedis fetchJedisConnector() {
    try {
        if (usePool && pool != null) {
            return pool.getResource();
        }
        Jedis jedis = new Jedis(getShardInfo());
        // force initialization (see Jedis issue #82)
        jedis.connect();
        return jedis;
    } catch (Exception ex) {
        throw new RedisConnectionFailureException("Cannot get Jedis connection", ex);
    }
}

After pool.getResource() the thread entered a waiting state. The pool implementation contains:

public T getResource() {
    try {
        return internalPool.borrowObject();
    } catch (Exception e) {
        throw new JedisConnectionException("Could not get a resource from the pool", e);
    }
}

Further inspection of borrowObject showed a loop that waits indefinitely when borrowMaxWaitMillis < 0L, which was the case because the configuration lacked a maxWaitMillis setting.

Using Alibaba’s Java diagnostic tool Arthas , the thread command displayed many http-nio threads stuck in WAITING state, confirming that all HTTP requests were blocked on the Redis connection acquisition.

The stack trace showed RedisConnectionFailureException: Cannot get Jedis connection repeatedly, confirming the root cause.

To fix the issue the Redis pool configuration was updated to set a reasonable timeout:

JedisConnectionFactory jedisConnectionFactory = new JedisConnectionFactory();
JedisPoolConfig config = new JedisPoolConfig();
config.setMaxWaitMillis(2000);
jedisConnectionFactory.setPoolConfig(config);
jedisConnectionFactory.afterPropertiesSet();

After restarting the service the problem re‑occurred, and further logs showed HTTP 500 errors caused by the same connection‑pool exhaustion.

The underlying bug was that stringRedisTemplate.getConnectionFactory().getConnection() was used directly, which obtains a Jedis connection without ever releasing it back to the pool. This left connections in a leased state, eventually exhausting the pool.

Recommended approaches are to use the Spring‑provided callback API, which ensures proper release:

stringRedisTemplate.execute(new RedisCallback() {
    @Override
    public Cursor doInRedis(RedisConnection connection) throws DataAccessException {
        return connection.scan(options);
    }
});

Or explicitly release the connection after use:

RedisConnectionUtils.releaseConnection(conn, factory);

Additionally, avoid using the KEYS command in production and configure the Redis pool size and timeout appropriately to prevent silent deadlocks.

Summary: Spring’s StringRedisTemplate abstracts most Redis operations but does not support commands like SCAN or SETNX directly; obtaining a raw Jedis connection without proper release can cause pool exhaustion and API hangs. Proper configuration of maxWaitMillis and using Spring’s callback utilities prevent these issues.