How LoongSuite Enables Full‑Stack Observability for LLM Applications

Foundational Model Breakthroughs

Domestic large models such as DeepSeek and Qwen have closed the performance gap with OpenAI and Anthropic by scaling parameters, improving inference speed, and adding multimodal capabilities. Development is shifting from pure performance to vertical specialization, creating a dual‑track of general‑purpose and industry‑specific models.

Full‑Stack Development Frameworks

Python remains the dominant language for AI development. High‑code frameworks like LangChain and LlamaIndex provide chain processing and knowledge‑base management. Java’s Spring AI (Alibaba) now offers comparable features, and low‑code platforms such as Dify and Coze enable visual orchestration and lightweight deployment. These frameworks are typically combined with MLOps tools, vector databases, and other infrastructure to close the development‑operations loop.

Typical AI‑Native Application Architecture

Requests from browsers, mini‑programs, Android, and iOS are routed through a unified gateway (e.g., Higress) that performs security and traffic‑control before forwarding to model‑application services written in Python, Java, or Go. Multiple model instances are deployed for high availability and cost‑aware load balancing, often behind a proxy such as Higress AI gateway.

Observability Requirements

Because the execution chain spans many components, a failure in any node can render the service unavailable. Full‑link diagnostics must pinpoint whether the issue lies in the AI application layer, the model inference layer, or downstream resources. An observability platform should correlate trace data with metrics (GPU utilization, token consumption, latency) to identify the root cause.

LoongSuite Data‑Collection Engine

LoongSuite provides a host‑level probe combined with process‑level instrumentation for Java, Go, and Python. The engine captures fine‑grained data (function entry/exit, parameters, execution time, resource usage) without modifying source code, structures the data, and forwards it to OpenTelemetry‑compatible back‑ends or proprietary analysis services.

Non‑Intrusive Instrumentation for Python

Python’s monkey‑patch mechanism is used to wrap target functions with a proxy that records input, output, execution time, and resource usage. This decorator‑style approach requires no source‑code changes.

Non‑Intrusive Instrumentation for Go

Go binaries cannot be altered at runtime, so LoongSuite leverages go‑toolexec to inject instrumentation between the compiler front‑end and back‑end. By modifying the abstract syntax tree (AST) according to predefined rules, LoongSuite inserts spans, metrics, and context‑propagation code during compilation, preserving the original binary’s behavior.

Dify Platform Architecture

Frontend (WebUI) : Nginx serves static assets and UI requests.

Backend API : Flask application handles configuration, chat sessions, and data management.

PostgreSQL : Persists application configs, conversation logs, and multi‑tenant data.

Redis : Caches temporary data and acts as a Celery message queue for asynchronous tasks.

Object Storage : Stores uploaded files, knowledge‑base documents, and certificates (local or OSS).

Plugin Architecture (dify‑plugin‑daemon)

Manages plugin lifecycle, including model and tool calls via SSE.

Supports dynamic loading of local or remote plugins.

Enables bidirectional calls to Dify internal services (models, tools, agent strategies, extensions, bundles).

Deploying Dify with LoongSuite

Clone the dify‑plugin‑daemon repository.

Modify its Dockerfile to add the LoongSuite agent binary and install required Python dependencies (e.g., uv, tiktoken).

Build the enhanced image:

docker build -t my-dify-image -f docker/local.dockerfile .

Replace the original dify‑plugin‑daemon image in the deployment manifest with the newly built image.

Add a few label entries (as described in LoongSuite documentation) to enable automatic telemetry collection.

Full‑Link Tracing Results

After integration, the Dify console displays unified monitoring data for both the Python‑based dify‑api service and the Go‑based dify‑plugin‑daemon component. The end‑to‑end call chain is correlated, demonstrating LoongSuite’s ability to provide comprehensive observability across heterogeneous LLM services.

References

Alibaba OpenTelemetry Go Auto‑Instrumentation: https://github.com/alibaba/opentelemetry-go-auto-instrumentation

LoongSuite Go Agent Repository: https://github.com/alibaba/loongsuite-go-agent

Dify Plugin Daemon Repository: https://github.com/langgenius/dify-plugin-daemon