Diagnosing and Resolving Redis Connection Pool Blocking in a Spring Boot Application
The article details a week‑long investigation of a sandbox environment where Spring Boot APIs became unresponsive due to Redis connection pool blocking, describing the use of system tools, JVM thread analysis, and code inspection to identify mis‑configured pool settings and recommending proper connection handling with Spring's RedisTemplate.
The team observed that internal sandbox APIs repeatedly hung, with no errors in the database or Redis logs. Initial attempts to restart the application only provided temporary relief, prompting a deeper investigation using top , jstack , and the Alibaba Arthas diagnostic tool.
Thread dumps revealed many http-nio Tomcat threads stuck in a waiting state, and further analysis of the Redis connection pool showed that calls to pool.getResource() were blocking because the pool's borrowObject method entered an endless wait when borrowMaxWaitMillis was negative or unset.
Key code excerpts that caused the blockage include:
protected Jedis fetchJedisConnector() {
try {
if (usePool && pool != null) {
return pool.getResource();
}
Jedis jedis = new Jedis(getShardInfo());
jedis.connect();
return jedis;
} catch (Exception ex) {
throw new RedisConnectionFailureException("Cannot get Jedis connection", ex);
}
} public T getResource() {
try {
return internalPool.borrowObject();
} catch (Exception e) {
throw new JedisConnectionException("Could not get a resource from the pool", e);
}
} public T borrowObject(long borrowMaxWaitMillis) throws Exception {
this.assertOpen();
// ... omitted for brevity ...
while (p == null) {
if (blockWhenExhausted) {
p = (PooledObject) this.idleObjects.pollFirst();
if (p == null) {
create = true;
p = this.create();
}
if (p == null) {
if (borrowMaxWaitMillis < 0L) {
p = (PooledObject) this.idleObjects.takeFirst();
} else {
long waitTime = System.currentTimeMillis();
p = (PooledObject) this.idleObjects.pollFirst(borrowMaxWaitMillis, TimeUnit.MILLISECONDS);
waitTime = System.currentTimeMillis() - waitTime;
}
if (p == null) {
throw new NoSuchElementException("Timeout waiting for idle object");
}
}
}
}
return p;
}After identifying that the pool lacked a proper maxWaitMillis configuration, the developers added the setting (e.g., config.setMaxWaitMillis(2000) ) and ensured connections were released using RedisConnectionUtils.releaseConnection(conn, factory) or by executing commands via stringRedisTemplate.execute with a RedisCallback .
The final recommendation is to avoid directly obtaining raw connections from stringRedisTemplate.getConnectionFactory().getConnection() , configure the Redis pool appropriately, and prefer Spring's higher‑level abstractions for safe resource management.
Architecture Digest
Focusing on Java backend development, covering application architecture from top-tier internet companies (high availability, high performance, high stability), big data, machine learning, Java architecture, and other popular fields.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.