Mastering API Rate Limiting in Go: Practical Algorithms and Implementation

Why Rate Limiting Matters in Go Services

In many Go projects, rate limiting is added only after the system shows latency spikes, jitter, or is hit by abnormal traffic, revealing that limiting is not just a simple guard but a classic algorithmic engineering problem.

1. Naïve "If Count > Limit" Check (Bad Example)

if reqCount > limit {
    return errors.New("rate limited")
}

Advantages: low implementation cost, intuitive logic, works for early prototypes. Drawbacks: no time dimension, count only increases, cannot recover, and provides no tuning space once traffic fluctuates.

2. Fixed Window Limiter

Core idea: divide time into fixed‑length windows, count requests per window, reject when the count exceeds the threshold.

Complete Fixed Window Implementation

import "sync/atomic"

type SimpleLimiter struct {
    limit int64
    count int64
}

func NewSimpleLimiter(limit int64) *SimpleLimiter {
    return &SimpleLimiter{limit: limit}
}

func (l *SimpleLimiter) Allow() bool {
    if atomic.AddInt64(&l.count, 1) > l.limit {
        return false
    }
    return true
}

This version suffers from three fatal issues: no time dimension, count never decrements, and it cannot reset, making it unsuitable for stable services.

3. Sliding Window Limiter

To overcome fixed‑window boundary effects, the sliding window continuously tracks the request count over the most recent time interval.

Always count requests in the "most recent period".

Engineered Sliding Window Implementation

import "sync"
import "time"

type SlidingWindowLimiter struct {
    limit       int
    bucketCount int
    buckets     []int
    lastTime    int64
    mu          sync.Mutex
}

func NewSlidingWindowLimiter(limit int, windowSize int) *SlidingWindowLimiter {
    return &SlidingWindowLimiter{
        limit:       limit,
        bucketCount: windowSize,
        buckets:     make([]int, windowSize),
        lastTime:    time.Now().Unix(),
    }
}

func (l *SlidingWindowLimiter) Allow() bool {
    l.mu.Lock()
    defer l.mu.Unlock()

    now := time.Now().Unix()
    diff := now - l.lastTime
    if diff > 0 {
        if diff >= int64(l.bucketCount) {
            for i := range l.buckets {
                l.buckets[i] = 0
            }
        } else {
            for i := int64(1); i <= diff; i++ {
                index := (l.lastTime + i) % int64(l.bucketCount)
                l.buckets[index] = 0
            }
        }
        l.lastTime = now
    }

    total := 0
    for _, c := range l.buckets {
        total += c
    }
    if total >= l.limit {
        return false
    }

    index := now % int64(l.bucketCount)
    l.buckets[index]++
    return true
}

Advantages: smoother traffic statistics, predictable behavior; requires careful selection of bucket count based on QPS.

4. Token Bucket Limiter

The token‑bucket model allows short bursts while enforcing an overall rate, making it ideal for user‑experience‑sensitive APIs.

Practical Token Bucket Implementation

import "sync"
import "time"

type TokenBucket struct {
    capacity int64
    tokens   int64
    rate     int64
    lastTime int64
    mu       sync.Mutex
}

func NewTokenBucket(capacity, rate int64) *TokenBucket {
    return &TokenBucket{capacity: capacity, tokens: capacity, rate: rate, lastTime: time.Now().Unix()}
}

func (b *TokenBucket) Allow() bool {
    b.mu.Lock()
    defer b.mu.Unlock()

    now := time.Now().Unix()
    elapsed := now - b.lastTime
    if elapsed > 0 {
        newTokens := elapsed * b.rate
        if newTokens > 0 {
            b.tokens = min(b.capacity, b.tokens+newTokens)
            b.lastTime = now
        }
    }
    if b.tokens <= 0 {
        return false
    }
    b.tokens--
    return true
}

func min(a, b int64) int64 {
    if a < b { return a }
    return b
}

Suitable scenarios: APIs sensitive to user experience, services that can tolerate short bursts, and cases where overall request rate must be controlled.

5. Practical Engineering Tips for Go Rate Limiting

✅ Ensure Concurrency Safety

Count and time updates must be atomic.

Avoid coarse‑grained locks; consider atomic ops or sharded locks under high concurrency.

⚙️ Tune Parameters Over Algorithm Names

windowSize : length of the time window.

bucketCount : granularity of statistics.

rate : token generation speed (for token bucket).

Choosing the wrong parameters renders even the best algorithm ineffective.

📍 Place Limiter at the Right Layer

API entry point.

Middleware.

Gateway layer.

The limiter should fit the overall system architecture; earlier is not always better.

6. Choosing the Right Algorithm – An Engineering Decision

Fixed Window – Simple, low memory, but suffers from burst spikes at window boundaries.

Sliding Window – Smooths traffic, handles bursts, slightly more complex and memory‑intensive.

Token Bucket – Allows controlled bursts, best for public gateways, but has more parameters and implementation complexity.

Algorithmic value lies in making system behavior controllable and predictable, not in code complexity.

Conclusion

Rate limiting is a foundational safeguard for services facing unpredictable traffic. Designing a proper limiter marks the transition from a throw‑away guard to a production‑grade algorithmic solution.