Boost Go Performance: Enhancing singleflight for Read/Write Concurrency

Hello, I'm Xiao Lou. Recently I enhanced Go's singleflight package to solve a performance issue, and I'm sharing the details.

What is singleflight?

singleflight suppresses duplicate requests, ensuring that at any moment only one identical request is executed while others wait and receive the same result.

For example, when many concurrent reads (Redis, MySQL, HTTP, RPC) occur, singleflight limits concurrency to one.

How singleflight works

It originated in the groupcache project and later moved into the Go standard library. The core implementation defines a call struct with a WaitGroup, a value, and an error.

type call struct {
    wg  sync.WaitGroup
    val interface{}
    err error
}

A global Group holds a mutex and a map from string keys to *call.

type Group struct {
    mu sync.Mutex
    m  map[string]*call
}

The Do method takes a key and a function to execute. If a call for the same key exists, it waits; otherwise it creates a new call, runs the function, stores the result, and cleans up.

func (g *Group) Do(key string, fn func() (interface{}, error)) (interface{}, error) {
    g.mu.Lock()
    if g.m == nil {
        g.m = make(map[string]*call)
    }
    if c, ok := g.m[key]; ok {
        g.mu.Unlock()
        c.wg.Wait()
        return c.val, c.err
    }
    c := new(call)
    c.wg.Add(1)
    g.m[key] = c
    g.mu.Unlock()

    c.val, c.err = fn()
    c.wg.Done()

    g.mu.Lock()
    delete(g.m, key)
    g.mu.Unlock()
    return c.val, c.err
}

The logic can be split into three parts: lazy map initialization, waiting for existing calls, and creating a new call when none exists.

① Lazy initialization of the map.

② If the key exists, wait for the ongoing call and reuse its result.

③ If the key does not exist, create a call, execute the function, then clean up.

Can we suppress writes as well?

singleflight works well for concurrent reads, but handling concurrent writes (e.g., service registration events) requires a different approach.

In a microservice registration center, many registration events can burst, causing performance issues.

Interface-level registration (like Dubbo) where each machine registers many times.

Concurrent service releases, e.g., restarting 100 machines simultaneously.

One naive solution is to merge pushes, but timing and freshness become concerns.

Directly using singleflight?

Applying singleflight to event pushes would hold subsequent identical events until the first completes, potentially discarding newer data versions.

Small enhancement

Introduce a WriteGroup that batches subsequent events after the first completes, ensuring later events are also processed.

Enhanced code reference

The WriteGroup reuses the original singleflight logic, adding a map of WaitGroups for write coordination.

package singleflight

import (
    "sync"
)

type WriteGroup struct {
    mu    sync.Mutex
    wgs   map[string]*sync.WaitGroup
    group Group
}

func (g *WriteGroup) Do(key string, fn func() error) error {
    g.mu.Lock()
    if g.wgs == nil {
        g.wgs = make(map[string]*sync.WaitGroup)
    }
    wg, ok := g.wgs[key]
    if !ok {
        wg = &sync.WaitGroup{}
        wg.Add(1)
        g.wgs[key] = wg
    }
    g.mu.Unlock()

    if !ok {
        err := fn()
        g.mu.Lock()
        wg.Done()
        delete(g.wgs, key)
        g.mu.Unlock()
        return err
    }

    wg.Wait()
    _, err := g.group.Do(key, func() (interface{}, error) {
        return nil, fn()
    })
    return err
}

Results

When concurrency exceeds two, the enhancement merges events effectively; in load tests with 1500 concurrent registrations, 99.9% of events were merged, delivering dramatic performance gains.