How High-Watermark Throttling Saves MySQL in Flash‑Sale Scenarios

Flash‑sale (秒杀) workloads, originating from large‑scale promotional events like Alibaba's Double‑11, involve three high‑intensity characteristics: massive instantaneous concurrency, strict data consistency, and frequent hotspot updates. These conditions cause severe database pressure, leading to lock contention, performance degradation, and even avalanche failures, as illustrated by a TV station’s failed red‑packet distribution during a Spring Festival.

To mitigate these issues, Alibaba has introduced a set of optimizations, focusing here on the underlying data‑layer technique called high‑watermark throttling . The method monitors the MySQL status variable threads_running and, when it exceeds a configurable threshold, rejects further SQL execution with a clear error message ("MySQL Server is too busy"). This protects the database from being overwhelmed by sudden spikes.

Implementation Details

threads_running_ctl_mode : Determines which SQL types are subject to throttling. Options are ALL or SELECTS (default). Use ALL with caution.

threads_running_high_watermark : The threshold value. By default it equals max_connections; setting it to 0 resets it to max_connections.

Throttling is triggered only when both conditions are met: threads_running exceeds threads_running_high_watermark.

The SQL type matches the mode defined by threads_running_ctl_mode.

Exceptions (requests that are not rejected) include:

Users with SUPER privileges.

SQL statements already inside an open transaction.

Commit or rollback statements.

Production Case Study

Alibaba Cloud RDS integrates these parameters. Enabling them in a scenario with heavy concurrent updates to the same row dramatically stabilizes performance. The following figures (originally from the source) illustrate the impact:

Before throttling, active connections peaked at 10,000 and update TPS was highly unstable.

After enabling the high‑watermark limit, active connections dropped to around 300 and TPS stabilized, effectively protecting the database.

Conclusion

High‑watermark throttling provides a simple yet powerful safeguard for MySQL databases under flash‑sale workloads, preventing lock contention and performance collapse by capping concurrent thread execution. Proper configuration of threads_running_ctl_mode and threads_running_high_watermark allows operators to balance throughput and stability.