Mastering Microservice Rate Limiting: Strategies, Algorithms, and TSF Implementation

Why rate limiting matters

In high‑concurrency microservice environments, sudden traffic spikes can trigger service avalanches. Rate limiting is a core fault‑tolerance technique that protects system stability by controlling request rates.

Considerations before applying rate limiting

Objectives

Prevent overload – keep the system stable under heavy load.

Enhance security – limit login, promo‑code, or other sensitive endpoints to mitigate brute‑force attacks.

Guarantee service quality – ensure fair access for all users.

Control operational cost – avoid uncontrolled scaling that leads to excessive expenses.

Design principles

Fairness – all callers should receive equal treatment.

Flexibility – support multiple dimensions (IP, user ID, API) and allow relaxed limits during peak periods.

Decoupling – keep rate‑limit logic separate from business code for easier maintenance.

Observability – expose limit rules and current status to users for transparency.

Limiting targets

Single‑node limits using fixed or sliding windows.

Cluster limits backed by a distributed store such as Redis.

Business‑object limits (per‑IP, per‑user, per‑API, etc.).

Common rate‑limit algorithms

Fixed‑window counter

Counts requests within a fixed time slot (e.g., per minute). The counter resets at the start of each slot.

Pros : Simple to implement and understand.

Cons : Can cause traffic bursts at slot boundaries; not smooth for sudden spikes.

Typical use : Scenarios with relatively uniform request distribution, such as PV/UV statistics.

Sliding‑window counter

Divides time into many small sub‑windows and aggregates counts across them, providing smoother traffic control.

Pros : Better smoothing of bursts; finer‑grained control.

Cons : More complex implementation; higher memory and CPU overhead.

Typical use : Environments with unpredictable burst traffic.

Leaky‑bucket (漏桶)

Models a fixed‑capacity queue that drains at a constant rate, smoothing bursts regardless of arrival pattern.

Pros : Guarantees a steady output rate; effective for traffic shaping.

Cons : Limited flexibility for burst handling; requires bucket state management.

Typical use : Network traffic shaping, API throttling, database protection.

Token‑bucket

Allows a configurable burst size while maintaining an average rate. Tokens are added at a fixed rate and consumed per request.

Pros : Supports bursts and smooth average flow.

Cons : More complex state handling and synchronization.

Typical use : High‑traffic APIs, flash‑sale or hot‑news scenarios with bursty workloads.

Post‑limit actions

Reject new requests with HTTP 429.

Block and wait briefly before retrying.

Adjust load‑balancer weights to divert traffic.

Log throttled requests for analysis.

Rate limiting in Tencent Service Framework (TSF)

TSF overview

TSF is a PaaS platform that provides full lifecycle management, observability, and service governance for microservices. It supports Spring Cloud and Service Mesh.

Fundamentals

Granularity : Configurable via tag expressions (global, per‑service, per‑API, per‑caller).

Threshold : QPS‑based limits (requests per second).

Status : Rules can be enabled or disabled.

Global vs. tag‑based limiting

Global : Counts all traffic to a service regardless of source.

Tag‑based : Uses label expressions to differentiate callers or APIs, enabling fine‑grained control.

Implementation flow

Configure a limit rule in the TSF console.

TSF calls the yunapi interface.

The request passes through tsf-dispatch and reaches the tsf‑ratelimit component. tsf‑ratelimit‑master aggregates historical traffic from all instances, predicts future load, and distributes per‑instance quotas.

Each instance’s SDK applies a token‑bucket algorithm using the received quota.

TSF token‑bucket details

Tokens are added at rate r (tokens per second).

Bucket capacity is b; excess tokens are discarded.

When a request of size n bytes arrives, n tokens are removed. If insufficient tokens exist, the request is delayed, dropped, or marked.

The bucket permits bursts up to b bytes while enforcing a long‑term average rate r.

Typical TSF use cases

Limit based on caller identity (global vs. specific upstream service).

Limit specific APIs or exclude certain APIs.

Combine multiple rules for complex protection scenarios.

Complementary governance

Rate limiting should be combined with circuit breaking, degradation, and other resilience patterns to handle downstream failures and prevent cascading overloads.