How STAROps Detects Unknown Anomalies with Intelligent Log Inspection

Most engineers query logs only after an alert fires—searching by request ID, pod restart, or user‑reported failure—so logs help reconstruct the incident after it happens. However, logs also capture business execution, traffic quality, audit trails, and Kubernetes events, providing a rich source for both troubleshooting and proactive anomaly detection.

Unknown changes are easy to miss: a new version introduces a rarely seen stack trace, error rates appear normal overall but a specific region degrades, or high‑risk operations occur outside authorized windows. These signals may seem insignificant alone, yet they can precede large‑scale failures.

STAROps addresses this by turning passive log review into active inspection. The system continuously scans logs, flags any pattern that has never appeared before, and surfaces it for immediate investigation.

SLS, Log Clustering, and Dimension Drill‑Down

The workflow begins with SLS (Log Service) performing large‑scale analysis. AI operators then:

Cluster logs into Pattern templates by extracting stable constants and dynamic variables.

Compare pattern counts and variable distributions across windows to spot new, vanished, or surging patterns.

Group clusters by service, module, or other fields and directly contrast the current window with yesterday, last week, or pre‑release baselines.

Example log snippet:

1WARN  [worker-17] Connection pool exhausted, tenant=shop-102, wait=812ms
2ERROR [gateway] Downstream timeout, provider=bank-a, api=/pay/confirm, elapsed=3201ms
3WARN  [worker-08] Connection pool exhausted, tenant=shop-518, wait=943ms

Pattern A: <*> WARN [<*>] Connection pool exhausted, tenant=<*>, wait=<*>ms
Pattern B: <*> ERROR [gateway] Downstream timeout, provider=<*>, api=<*>, elapsed=<*>ms

The Agent sees the overall distribution of patterns in the analysis window, noting how often each pattern occurs, which are new or sharply increasing, and how variable values are distributed. Even without a predefined rule, a newly emerging pattern like "Connection pool exhausted" can be detected.

Dimension Drill‑Down

For each window the system builds a baseline and then examines dimensions such as service, region, version, and operation. It identifies the dimension range with the most significant deviation and reports actual vs. baseline values. This approach is more efficient than naïvely grouping by every field, because the highest‑traffic dimension is not always the root cause.

When a single dimension cannot explain a change, the system explores combinations (e.g., project-a + logstore-x) to isolate the anomaly.

Dynamic Planning and Precise Drill‑Down

During the first run, STAROps generates an initial plan based on log structure, history, and inspection goals. In subsequent rounds the Agent reviews evidence, confirms comparability with baselines, and refines the query scope—e.g., locking onto project-a and then narrowing to specific logstores and versions. Each iteration’s findings are stored in a Finding record that preserves identifiers, evidence, impact range, and status.

Findings drive notification policies: only when a Finding’s error rate exceeds 5 % or a new critical pattern appears does STAROps push alerts to DingTalk or Feishu; otherwise reports remain silent.

Extending the Inspection with Skills and UModel

Custom probes can be added to focus on business‑critical windows, unauthorized high‑risk actions, or specific error codes. Skill modules and Runbook scripts encode domain knowledge—e.g., after detecting a new Java stack trace, a Skill queries the code repository via MCP (Model Context Protocol) for the related file, commit, and release version.

UModel (Unified Model) maps entities such as services, pods, agents, Skills, metrics, traces, and logs onto a unified graph, allowing the inspection to traverse across data sources without moving raw data. The evidence chain follows the pattern: operator discovers candidate → Agent verifies → UModel fetches cross‑source evidence.

Continuous Feedback and Optimization

Each inspection run feeds back into the plan. For example, after the first run showed a 147 % increase in log volume concentrated in a batch task window, the team added business semantics (Skill and employeeId fields) to the plan. The next run revealed a p90 latency jump from 94 s to 171 s, but Skill‑level analysis showed the slowdown was due to traffic composition, not service degradation.

When non‑completion rates rose from 1.01 % to 6.2 %, the system sent a targeted alert with per‑Skill error distribution and identified the root cause as a context‑canceled error.

Key Takeaways

STAROps starts with a single Logstore and a generic plan, then iteratively enriches the inspection with business semantics, adjusts checklists and notification thresholds, and expands the investigation scope via Skills and UModel. Over time, unknown problems become reusable knowledge, noise is filtered out, and genuine changes are detected earlier, investigated faster, and reported with higher relevance.