Why Short Connections, Frequent INFO Calls, and Pipelines Hurt Redis Performance (and How to Fix It)

Zhang Pengyi, a senior engineer at Tencent Cloud Database, shares practical insights from his work on Redis performance optimization.

1. Short Connections Cause High CPU

Users often notice elevated CPU usage on the redis‑server when many short connections are created. Profiling with perf revealed that the function listSearchKey dominates CPU time during connection release.

Benchmark comparison using redis-benchmark:

./redis-benchmark -h host -p port -t ping -c 10000 -n 500000 -k 1   # long connections (k=1)

Result (long connections):

PING_INLINE: 92902.27 requests per second
PING_BULK: 93580.38 requests per second

When the same test is run with short connections ( -k 0), QPS drops dramatically and listSearchKey becomes the top CPU consumer:

./redis-benchmark -h host -p port -t ping -c 10000 -n 500000 -k 0   # short connections
PING_INLINE: 15187.18 requests per second
PING_BULK: 16471.75 requests per second

Short connections add network overhead and force the server to perform extra cleanup work on each disconnect, turning an O(N) client‑list traversal into a noticeable CPU bottleneck.

2. Optimizing Connection Release

The original freeClient and unlinkClient functions locate the client in server.clients via listSearchKey, an O(N) operation. By storing a direct pointer to the client’s list node ( client_list_node) when the client is created, the removal can be performed in O(1).

void freeClient(client *c) {
    /* ... */
    sdsfree(c->querybuf);
    sdsfree(c->pending_querybuf);
    /* ... */
    if (c->flags & CLIENT_BLOCKED) unblockClient(c);
    dictRelease(c->bpop.keys);
    unwatchAllKeys(c);
    listRelease(c->watched_keys);
    pubsubUnsubscribeAllChannels(c,0);
    pubsubUnsubscribeAllPatterns(c,0);
    dictRelease(c->pubsub_channels);
    listRelease(c->pubsub_patterns);
    listRelease(c->reply);
    freeClientArgv(c);
    /* unlink client */
    unlinkClient(c);
}

void unlinkClient(client *c) {
    if (server.current_client == c) server.current_client = NULL;
    if (c->fd != -1) {
        aeDeleteFileEvent(server.el,c->fd,AE_READABLE);
        aeDeleteFileEvent(server.el,c->fd,AE_WRITABLE);
        close(c->fd);
        c->fd = -1;
    }
    /* O(1) removal using stored node */
    listDelNode(server.clients, c->client_list_node);
}

Client creation is also modified to record the node:

client *createClient(int fd) {
    client *c = zmalloc(sizeof(client));
    /* ... */
    if (fd != -1) {
        c->client_list_node = listAddNodeTailEx(server.clients,c);
    }
    initClientMultiState(c);
    return c;
}

3. INFO Command Increases CPU

Frequent execution of the INFO command forces the server to iterate over all clients to compute statistics such as client_longest_output_list and used_memory_overhead. Profiling shows functions like getClientsMaxBuffers and getMemoryOverheadData consuming significant CPU.

Sample INFO output (truncated):

client
connected_clients:1
client_longest_output_list:0
client_biggest_input_buf:0
Memory
used_memory:848392
used_memory_human:828.51K
used_memory_rss:3620864
used_memory_overhead:836182

Two experiments illustrate the impact:

One persistent connection continuously issuing INFO consumes ~20% CPU.

9999 idle connections plus one connection looping INFO pushes CPU usage to ~80%.

Go code for the first experiment:

func main() {
    c, err := redis.Dial("tcp", addr)
    if err != nil { fmt.Println("Connect error:", err); return }
    for {
        c.Do("info")
    }
}

Go code for the second experiment (many idle connections + one active INFO loop) is analogous.

4. Pipeline Can Increase Memory

When using pipelines for read‑only operations, Redis buffers the replies until the event loop flushes them. If the client does not promptly read the replies, the server’s reply buffer grows, leading to high memory usage.

Go pipeline example:

c, _ := redis.Dial("tcp", "127.0.0.1:6379")
 c.Send("get", "key1")
 c.Send("get", "key2")
 c.Send("get", "key3")
 c.Flush()
 fmt.Println(redis.String(c.Receive()))
 fmt.Println(redis.String(c.Receive()))
 fmt.Println(redis.String(c.Receive()))

On the server side, the pseudo‑code for handling pipelined commands shows that each command is parsed, executed, its reply appended to replyBuffer, and the client is marked pending write. The buffer is only sent later, so a slow or buggy client can cause the buffer to retain many replies, inflating memory.

readQueryFromClient(client* c) {
    read(c->querybuf);
    cmdsNum = parseCmdNum(c->querybuf);
    while(cmdsNum--) {
        cmd = parseCmd(c->querybuf);
        reply = execCmd(cmd);
        appendReplyBuffer(reply);
        markClientPendingWrite(c);
    }
}

5. Summary and Recommendations

Avoid short‑lived connections; prefer persistent connections or connection pooling.

Do not execute INFO frequently in high‑connection scenarios.

When using pipelines, read replies promptly and limit the number of commands per pipeline batch.