Why Did Our Spring API Hang? Uncovering Redis Connection Pool Blocking

In an internal sandbox environment the API became unresponsive for a whole week. Initial attempts to restart the application temporarily restored service, but the problem recurred more frequently.

Using top -H -p 12798 we observed that the machine itself was healthy, so we inspected the JVM stack with jstack. The most CPU‑intensive threads were waiting on a lock.

Further investigation with top -H -p 12798 confirmed the three threads consuming the most resources. We then used jstack 12798 to view the stack traces.

The stack showed that threads were blocked in pool.getResource() of the Jedis connection pool.

/**
 * Returns a Jedis instance to be used as a Redis connection. The instance can be newly created or retrieved from a
 * pool.
 */
protected Jedis fetchJedisConnector() {
    try {
        if (usePool && pool != null) {
            return pool.getResource();
        }
        Jedis jedis = new Jedis(getShardInfo());
        // force initialization (see Jedis issue #82)
        jedis.connect();
        return jedis;
    } catch (Exception ex) {
        throw new RedisConnectionFailureException("Cannot get Jedis connection", ex);
    }
}

Inside the pool, getResource() eventually calls borrowObject(), which contains a loop that waits indefinitely when borrowMaxWaitMillis is negative.

public T getResource() {
    try {
        return internalPool.borrowObject();
    } catch (Exception e) {
        throw new JedisConnectionException("Could not get a resource from the pool", e);
    }
}

public T borrowObject(long borrowMaxWaitMillis) throws Exception {
    this.assertOpen();
    // ...
    while (p == null) {
        boolean create = false;
        if (blockWhenExhausted) {
            p = (PooledObject) this.idleObjects.pollFirst();
            if (p == null) {
                create = true;
                p = this.create();
            }
            if (p == null) {
                if (borrowMaxWaitMillis < 0L) {
                    p = (PooledObject) this.idleObjects.takeFirst();
                } else {
                    long waitTime = System.currentTimeMillis();
                    p = (PooledObject) this.idleObjects.pollFirst(borrowMaxWaitMillis, TimeUnit.MILLISECONDS);
                    waitTime = System.currentTimeMillis() - waitTime;
                }
            }
            if (p == null) {
                throw new NoSuchElementException("Timeout waiting for idle object");
            }
        }
        // ...
    }
    return p;
}

The missing maxWaitMillis configuration caused the pool to wait forever, leading to all HTTP‑nio threads being stuck in a waiting state, as shown by the Arthas thread command.

After adding the missing configuration:

JedisPoolConfig config = new JedisPoolConfig();
config.setMaxWaitMillis(2000);
JedisConnectionFactory jedisConnectionFactory = new JedisConnectionFactory();
jedisConnectionFactory.setPoolConfig(config);
jedisConnectionFactory.afterPropertiesSet();

the issue persisted because the borrowed connections were never returned to the pool. The root cause was that stringRedisTemplate.getConnectionFactory().getConnection() was used directly without releasing the connection, leaving it in a non‑idle state.

Correct usage is to execute Redis commands via a RedisCallback so that the connection is automatically released, or manually release it with RedisConnectionUtils.releaseConnection(conn, factory) after use.

stringRedisTemplate.execute(new RedisCallback() {
    @Override
    public Cursor doInRedis(RedisConnection connection) throws DataAccessException {
        return connection.scan(options);
    }
});

RedisConnectionUtils.releaseConnection(conn, factory);

In addition, avoid using the KEYS command in production and configure the Redis pool size and timeout appropriately to prevent silent deadlocks.

Summary: The API hang was caused by a misconfigured Jedis connection pool that blocked threads waiting for a Redis connection; fixing the pool’s maxWaitMillis and ensuring connections are properly released resolves the issue.