How Douyin Quickly Shut Down Comments: Service Governance and Circuit Breaking in Action
When the low‑quality app "XX段子" was forced to close, Douyin faced a massive comment surge and responded within minutes by disabling the comment service through circuit‑breaking, illustrating the crucial role of microservice governance, monitoring, and rapid fault isolation in large‑scale backend systems.
On April 10, a low‑quality audio‑sharing app “XX段子” was ordered to shut down, causing a massive influx of users to Douyin’s comment sections.
Shortly after, Douyin disabled all comment functionality, showing empty comment lists despite the UI indicating thousands of comments.
Douyin later explained the outage was due to server maintenance and an upgrade, but the rapid response suggests a technical mechanism.
Based on public technical sharing from ByteDance, the company uses a Go‑based microservice framework called kite, which is fully compatible with Thrift. The framework provides service registration and discovery, distributed load balancing, timeout and circuit‑breaker management, service degradation, method‑level metrics, and distributed tracing, supporting unlimited horizontal scaling.
When the “XX段子” shutdown caused a surge in comment‑fetch requests, the monitoring system likely triggered an alarm. Engineers investigated the spike and decided to apply a circuit‑breaker to the comment service around 23:40 on April 10, causing all comment‑list requests to return empty data.
A circuit‑breaker automatically cuts off a service when a defined metric (e.g., request count exceeding 1 million per minute) reaches a threshold, returning a default response; it can also be triggered manually.
This incident demonstrates the importance of service governance—monitoring, alerting, and circuit‑breaking—in handling sudden traffic spikes and preventing larger system failures.
Beyond emergency response, robust service monitoring helps developers trace distributed calls and diagnose online issues, as illustrated by the author’s previous experience with a monitoring system that aggregates exception rates and displays full call chains.
In summary, effective service governance is essential for large‑scale products, and techniques like circuit‑breaking are a small but critical part of that toolbox.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
Programmer DD
A tinkering programmer and author of "Spring Cloud Microservices in Action"
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
