How Skywalking’s New AI Features Turn Observability into Intelligent Diagnosis

Introduction

Distributed tracing tools have long visualized call graphs, but the final step of interpreting spans, locating slow requests, and correlating alerts still required manual effort. Skywalking now adds a complete AI capability matrix that turns the tool from a pure visualizer into an intelligent diagnosis platform.

"The last mile of distributed tracing has finally been bridged by AI."

What AI Features Are Integrated?

Skywalking’s AI capabilities are organized into three distinct layers:

Horizon UI AI Assistant – a natural‑language query engine that reads real‑time observability data from the OAP backend and returns a readable report with live charts.

Virtual GenAI – a “digital dashboard” for large‑model (LLM) calls that exposes latency, token usage, estimated cost, and first‑token latency (TTFT) for each provider and model.

AI Pipeline – a machine‑learning‑driven module that automatically learns normal application behavior, generates smart baselines, detects anomalies, and performs root‑cause analysis without manual thresholds.

Horizon UI AI Assistant

Released with Horizon UI 1.0 (July 2026), the assistant lets users ask questions such as “Which services are unhealthy now?” or “Why is order‑service latency increasing?” The assistant queries the same OAP data as the dashboards, assembles a concise textual answer, and renders a real‑time chart directly in the UI. It reads three data sources: real‑time metrics via the OAP query protocol, Layer configurations that act as a knowledge base, and the user’s permission scope (read‑only).

The assistant is disabled by default; operators must enable it in the Horizon UI configuration and point it to a model endpoint (e.g., Ollama, OpenAI, Gemini). Example configuration:

ai:
  assistant:
    enabled: true
    model:
      endpoint: "http://your-llm-service:8000/v1"
      api_key: "your-api-key" # optional

Typical queries:

"Which services are currently unhealthy?"

"Why is order‑service slowing down?"

"Show me the P99 latency trend for the payment API."

Virtual GenAI

When a Java application or OTLP probe intercepts calls to popular AI frameworks (Spring AI, OpenAI SDK), Skywalking agents report trace data that follows a GenAI semantic model, including provider name, model name, input/output token counts, latency, and TTFT. The OAP extracts these fields and visualizes them on the GenAI dashboard.

The dashboard displays metrics at both provider and model granularity, such as CPM, SLA, latency percentiles (P50/P90/P99), token usage, TTFT, and estimated cost. Cost estimation is configured in gen-ai-config.yml:

providers:
- provider: openai
  prefix-match:
  - gpt
  models:
  - name: gpt-4o
    input-estimated-cost-per-m: 2.5
    output-estimated-cost-per-m: 10

Two monitoring modes are supported:

Gateway‑side monitoring via Envoy AI Gateway, which aggregates traffic from multiple providers.

Application‑side monitoring (Virtual GenAI) using the Java agent for per‑call granularity.

AI Pipeline

The pipeline communicates with third‑party AI services over gRPC and supports asynchronous processing of large telemetry streams. Core components include:

Baseline query service – intelligent baseline calculation for time‑series data.

HTTP URI recognition service – automatic pattern detection and classification of RESTful URLs using the R3 algorithm.

Prediction service – performance monitoring and evaluation of ML models.

Key AI functions:

Smart anomaly detection that learns normal behavior and alerts on deviations without manual thresholds.

Accelerated root‑cause analysis through multi‑dimensional metric correlation.

Adaptive baseline computation that distinguishes true anomalies from normal traffic cycles.

Intelligent URI pattern recognition for API usage insights.

Configuration example (enable in application.yml and set gRPC endpoint):

ai-pipeline:
  enabled: true
  grpc-endpoint: "your-ai-service:50051"

How to Enable Skywalking AI

1. Deploy Horizon UI

docker pull apache/skywalking-ui:horizon-latest
docker run -d -p 8080:8080 \
  -e OAP_BACKEND_URL=http://your-oap:12800 \
  apache/skywalking-ui:horizon-latest

2. Enable AI Assistant

Set enabled: true and configure the model endpoint as shown above.

3. Enable Virtual GenAI (Java applications)

Upgrade the Java agent to version ≥ 9.7, add the apm-toolkit-trace dependency (v9.7.0) to pom.xml, and configure cost estimation in gen-ai-config.yml.

4. Deploy and Configure AI Pipeline

Deploy a gRPC‑compatible AI service (e.g., an R3 implementation), enable the pipeline in application.yml, and turn on HTTP URI recognition.

Pros and Cons

Advantages

Natural‑language Q&A dramatically speeds up troubleshooting (20 min → ≈5 min).

LLM calls become fully observable with token usage, cost, and latency metrics.

Machine‑learning‑driven ops eliminate manual threshold tuning.

Layered adoption lets teams start with the most needed capability.

Model flexibility keeps data on‑premise and costs under control.

Active open‑source community ensures rapid evolution.

Drawbacks / Considerations

AI Assistant is off by default and requires manual enablement.

Virtual GenAI needs Java agent ≥ 9.7.

AI Pipeline requires an external AI service; it does not ship a model.

Cost estimation must be manually configured per model.

Some features (e.g., natural‑language to BydbQL) are still in preview.

Applicable Scenarios

Strongly recommended for daily on‑call troubleshooting, large‑model application development, multi‑provider LLM management, massive microservice operations, e‑commerce traffic planning, and resource‑utilization optimization.

Conclusion

Skywalking’s AI integration solves the “last mile” problem of distributed tracing by turning raw spans into actionable insights, making LLM calls transparent, and enabling proactive, predictive operations. The three‑layer stack—Horizon UI AI Assistant, Virtual GenAI, and AI Pipeline—collectively transforms Skywalking from a visualizer into an intelligent observability platform.

Resources

Apache Skywalking: https://github.com/apache/skywalking

Horizon UI: https://github.com/apache/skywalking-horizon-ui

Official documentation: https://skywalking.apache.org/docs/

Horizon UI AI Assistant announcement: https://skywalking.apache.org/blog/2026-07-06-horizon-ui-ai-assistant/

Virtual GenAI docs: https://skywalking.apache.org/docs/main/v10.4.0/en/setup/service-agent/virtual-genai/