Baidu Game Microservice Monitoring Practice and System Design

The article begins by highlighting the challenges developers face when handling urgent online incidents during holidays and the importance of a robust monitoring system for game microservices. It introduces Baidu's monitoring practice, which aims to help developers quickly identify and resolve issues.

In the early stage, Baidu leveraged Argus for machine and log monitoring, Monitor for business metrics, and SIA for visualization, but these solutions lacked a unified strategy and deep business integration, leading to delayed risk detection and inefficient problem localization.

The monitoring evolution addresses four major problems: delayed risk exposure, fragmented coverage, weak diagnostic capability, and alarm overload. A systematic design was introduced, focusing on risk control, intelligent monitoring, smart alarm, and efficient fault localization.

Risk control measures include automated test cases and release checks that reduce over 95% of deployment issues. Intelligent monitoring incorporates multi‑dimensional metrics (traffic, latency, SLA, revenue) and uses algorithms to detect periodic and non‑periodic anomalies.

Smart alarm design features hierarchical alerts, merging and filtering to reduce noise, and automatic escalation from email to SMS, phone calls, and robot notifications, ensuring critical alerts reach on‑call engineers.

Efficient fault localization combines trace links, robot notifications, and real‑time indexing via DataHub and DStream, enabling developers to pinpoint problematic code lines within minutes.

The article also presents a panoramic view of the monitoring ecosystem, detailing tools (Argus, SIA, robot alerts), metrics, and monitoring objects ranging from servers and logs to business data and core logic.

In the conclusion, the authors reflect on the benefits of systematic monitoring—improved timeliness, coverage, and debugging efficiency—and outline future goals such as automated fault handling and intelligent resource scaling.