Sogou’s AI‑Powered Ops: Smart Circuit Breaker, Fault Localization & Chatbot

1. Three Major Pain Points for Ops Engineers

Before discussing intelligent operations, Sogou identifies three common challenges: worry cost , insufficient intelligence , and annoyance cost .

1.1 Worry Cost

Ops teams monitor thousands of machines and dozens of metrics (response time, CPU, network I/O, disk I/O). Alerts can be overwhelming, and many do not reflect the real situation, leaving engineers constantly anxious about unknown failures.

1.2 Insufficient Intelligence

When complex incidents occur, engineers often cannot quickly locate the root cause, feeling that their “intelligence” is insufficient.

1.3 Annoyance Cost

At Sogou, engineers who do not engage in automation development are not promoted, forcing them to split time between reliability duties and development, which leads to frequent interruptions and low productivity.

2. Using “Intelligence” to Solve the Pain Points

Sogou adopts three AI‑driven solutions:

Intelligent circuit breaking for worry cost

Intelligent fault localization for insufficient intelligence

Intelligent Q&A chatbot “WeiMi” for annoyance cost

2.1 Intelligent Circuit Breaking

Ops data is a time‑series of metrics (response time, CPU, network, disk). With millions of series, manual monitoring is impossible. Sogou’s intelligent circuit‑breaker analyzes root causes and predicts failures, handling two main fault sources: code changes/deployments and infrastructure (data center, network, hardware). When abnormal metric changes are detected, the system can automatically stop or roll back services.

2.2 Intelligent Fault Localization

Sogou’s search architecture is highly complex. To quickly locate faults, the system extracts a request ID that propagates across modules, matches it against a set of rule templates, and determines the faulty component and node.

Rule‑hit statistics are visualized to make the final decision, and the knowledge base is continuously updated so that similar future incidents can be resolved automatically.

2.3 Intelligent Q&A Chatbot “WeiMi”

The chatbot, embedded in Sogou’s internal messaging tool, addresses the annoyance cost by automating three functions:

Smart ticket lookup : users input a ticket number and instantly see its status.

Smart person finder : if a query does not match the knowledge base, the bot identifies the relevant domain and recommends an expert.

Smart Q&A : the bot answers questions directly from the curated knowledge base.

By leveraging AI, Sogou reduces manual monitoring, speeds up fault resolution, and frees engineers to focus on higher‑value work, while laying the groundwork for future capabilities such as decision‑tree‑based root‑cause analysis, big‑data‑driven monitoring, and automated fault prediction.