Building a Redis Connection Pool in Go: Design and Implementation

The series "EasyRedis" builds a functional Redis service in 11 articles; this tenth article focuses on implementing a connection pool to reuse TCP connections when forwarding commands across distributed Redis nodes.

Pool structure definition

The pool holds an idles chan any buffer for idle objects. newObject creates a new resource; freeObject releases it. activeCount tracks how many objects have been created; when it reaches Config.MaxActive new creations block. waiting []chan any stores channels of goroutines waiting for a released object. closed bool indicates whether the pool has been shut down.

type Pool struct {
    Config
    newObject   func() (any, error)
    freeObject  func(x any)
    idles       chan any
    mu          sync.Mutex
    activeCount int
    waiting     []chan any
    closed      bool
}

func NewPool(new func() (any, error), free func(x any), conf Config) *Pool {
    if new == nil {
        logger.Error("NewPool argument new func is nil")
        return nil
    }
    if free == nil {
        free = func(x any) {}
    }
    p := Pool{Config: conf, newObject: new, freeObject: free, activeCount: 0, closed: false}
    p.idles = make(chan any, p.MaxIdles)
    return &p
}

Getting an object from the pool

Lock the pool with p.mu.Lock() to avoid race conditions.

Try to receive an idle object from idles; if none, call p.getOne() to create a new one. p.getOne() checks p.activeCount >= p.Config.MaxActive. If the limit is reached, it creates a waiting channel, appends it to waiting, unlocks, and blocks until an object is returned.

If the limit is not reached, it increments activeCount, unlocks, creates a new object via newObject, and returns it (or returns an error and rolls back the count).

func (p *Pool) Get() (any, error) {
    p.mu.Lock()
    if p.closed {
        p.mu.Unlock()
        return nil, ErrClosed
    }
    select {
    case x := <-p.idles:
        p.mu.Unlock()
        return x, nil
    default:
        return p.getOne()
    }
}

func (p *Pool) getOne() (any, error) {
    if p.activeCount >= p.Config.MaxActive {
        wait := make(chan any, 1)
        p.waiting = append(p.waiting, wait)
        p.mu.Unlock()
        x, ok := <-wait
        if !ok {
            return nil, ErrClosed
        }
        return x, nil
    }
    p.activeCount++
    p.mu.Unlock()
    x, err := p.newObject()
    if err != nil {
        p.mu.Lock()
        p.activeCount--
        p.mu.Unlock()
        return nil, err
    }
    return x, nil
}

Returning an object to the pool

Lock the pool.

If the pool is closed, release the object immediately via freeObject.

If there are waiting goroutines, pop the first waiting channel and send the object to unblock it.

Otherwise, try to place the object into the idles buffer; if the buffer is full, decrement activeCount and free the object.

func (p *Pool) Put(x any) {
    p.mu.Lock()
    if p.closed {
        p.mu.Unlock()
        p.freeObject(x)
        return
    }
    if len(p.waiting) > 0 {
        wait := p.waiting[0]
        temp := make([]chan any, len(p.waiting)-1)
        copy(temp, p.waiting[1:])
        p.waiting = temp
        wait <- x
        p.mu.Unlock()
        return
    }
    select {
    case p.idles <- x:
        p.mu.Unlock()
    default:
        p.activeCount--
        p.mu.Unlock()
        p.freeObject(x)
    }
}

Extending to a per‑IP socket connection pool

In real deployments each IP address gets its own pool. The RedisConnPool struct holds a concurrent dictionary mapping an address to a *pool.Pool. The pool is created with MaxIdles=1 and MaxActive=20.

type RedisConnPool struct {
    connDict *dict.ConcurrentDict // addr -> *pool.Pool
}

func NewRedisConnPool() *RedisConnPool {
    return &RedisConnPool{connDict: dict.NewConcurrentDict(16)}
}

func (r *RedisConnPool) GetConn(addr string) (*client.RedisClent, error) {
    var connectionPool *pool.Pool
    raw, ok := r.connDict.Get(addr)
    if ok {
        connectionPool = raw.(*pool.Pool)
    } else {
        newClient := func() (any, error) {
            cli, err := client.NewRedisClient(addr)
            if err != nil {
                return nil, err
            }
            cli.Start()
            if conf.GlobalConfig.RequirePass != "" {
                reply, err := cli.Send(aof.Auth([]byte(conf.GlobalConfig.RequirePass)))
                if err != nil {
                    return nil, err
                }
                if !protocol.IsOKReply(reply) {
                    return nil, errors.New("auth failed:" + string(reply.ToBytes()))
                }
            }
            return cli, nil
        }
        freeClient := func(x any) {
            if cli, ok := x.(*client.RedisClent); ok {
                cli.Stop()
            }
        }
        connectionPool = pool.NewPool(newClient, freeClient, pool.Config{MaxIdles: 1, MaxActive: 20})
        r.connDict.Put(addr, connectionPool)
    }
    raw, err := connectionPool.Get()
    if err != nil {
        return nil, err
    }
    conn, ok := raw.(*client.RedisClent)
    if !ok {
        return nil, errors.New("connection pool make wrong type")
    }
    return conn, nil
}

func (r *RedisConnPool) ReturnConn(peer string, cli *client.RedisClent) error {
    raw, ok := r.connDict.Get(peer)
    if !ok {
        return errors.New("connection pool not found")
    }
    raw.(*pool.Pool).Put(cli)
    return nil
}

The code demonstrates a complete, reusable connection‑pool mechanism suitable for distributed Redis clients written in Go.