How Large Language Models Are Revolutionizing Fault Localization

Background

In daily operations, frequent online incidents require rapid root‑cause identification to minimize user impact. Traditional manual troubleshooting is slow and error‑prone, especially when alerts flood monitoring channels.

Current Situation

Operators face three main challenges: overwhelming and scattered alert information, repetitive manual steps, and inconsistent handling due to varying experience.

Intelligent Learning in Fault Localization

3.1 Advantages of Large Models over Human Effort

Large models can process massive operational data, apply standardized analysis, and execute tasks at high speed, outperforming manual investigation.

3.2 Model‑Based Agent

An agent built on a large language model perceives its environment, reasons, and executes actions via defined tool functions. It uses function‑call APIs to fetch recent change logs, pod status, network alerts, and other metrics.

When a user asks, “Check the recent node status,” the agent selects the appropriate tool, executes it, and integrates the result.

3.3 Retrieval‑Augmented Generation (RAG) for Historical Annotations

RAG retrieves past annotated incidents from a knowledge base, providing probable root causes for similar alerts, thus guiding operators.

3.4 RAG + CoT Architecture

The combined architecture leverages a single agent with multiple tools, using CoT prompts to enforce execution order and stability, while RAG supplies historical context.

3.5 Process Flow

The workflow extracts domain and URL from alerts, queries application details, checks pod health, evaluates recent changes, inspects network and third‑party alerts, and finally consults the annotation system via RAG before aggregating a final diagnosis.

Architecture Upgrade

The existing single‑agent design suffers from token limits, tool‑selection ambiguity, alert latency, and opaque RAG processing.

The upgraded design introduces a supervisory brain (Supervisor) and multiple specialized teams (Agents) that collaborate to handle change detection, ingress checks, application status, and more, mitigating the previous drawbacks.

Future Outlook

As large language models mature, they will further eliminate pain points in operations fault diagnosis, offering real‑time, expert‑level insights, and moving the field toward proactive, automated, and highly reliable IT operations.