How a Next‑Gen Cloud‑Native Observability Platform Boosted Ticketing Stability by 80%

Customer Testimony

“By constructing a new‑generation observability system, we not only achieved a leap in stability but also established a data‑driven operations decision‑making mechanism, which has become the digital foundation for our product and business development.”

Background

Facing increasingly complex business systems and urgent stability challenges, a leading digital‑solution provider needed to improve fault discovery efficiency by 80% and cut operation costs by 40%.

Business Challenges

The company’s “1‑5‑10” stability goal (detect faults within 1 minute, locate root cause within 5 minutes, recover business within 10 minutes) exposed deep shortcomings in the existing monitoring system: blind spots in critical links, alarm storms causing fatigue, heavy reliance on manual experience for root‑cause analysis, fragmented logs and traces, and lack of a closed‑loop alarm management process.

High‑concurrency ticket‑sale scenarios caused data‑collection loss, storage delays, and query timeouts during peak traffic, while the heterogeneous observability stack (separate logs, metrics, traces) prevented an end‑to‑end view.

The previous APM solution was closed, offered limited extensibility, and its licensing model conflicted with the company’s rapid‑iteration, elastic needs.

Alibaba Cloud Solution – Building a Future‑Ready Observability System

The team designed a panoramic architecture based on Alibaba Cloud observability products, covering infrastructure to business applications.

Log Service (SLS) : real‑time collection, query and analysis of ticketing platform logs.

Application Real‑Time Monitoring Service (ARMS) : full‑link tracing and performance profiling.

Observability Monitoring (Prometheus) Edition : metric collection and alerting for containerized and cloud resources.

Cloud Probe : user‑perspective service availability and performance verification.

The integrated stack broke monitoring silos, forming a “perceive‑locate‑respond” closed‑loop.

Key Capability Construction

1) Basic Coverage : unified data collection, layered metric system, and foundational alerts covering code to user, application to infrastructure.

2) Metric System : detailed metrics for application (QPS, error rate, latency, JVM, thread pool), infrastructure (CPU, memory, disk I/O, network), and middleware (DB slow queries, Redis hit rate, MQ backlog) visualized in Grafana dashboards.

3) Intelligent Alarm System : ARMS alarm hub aggregates alerts from multiple sources, applies business‑priority routing, supports webhook integration, and enforces a full lifecycle from generation to recovery verification, linking alerts to SLOs for continuous improvement.

Collaborative Efficiency

Business‑centric health scores (transaction success rate, response time, system throughput) are quantified and visualized, mapping technical anomalies to concrete business impact (e.g., “estimated loss of X orders per minute”). The system integrates with a one‑stop development platform, allowing developers to view alerts, health scores, traces, and logs without leaving their portal.

Intelligent Operations (AI‑Driven)

An AI‑powered MCP workflow aggregates daily alarms, uses large‑model inference to cluster and attribute cross‑service issues, and generates concise reports with root‑cause hypotheses and actionable recommendations, reducing information overload.

Prompt‑engineered agents extract alarm context, feed it to the model, and produce standardized insights that feed both daily reports and weekly retrospectives.

Future Outlook

The roadmap includes finer data governance, deeper AIOps capabilities for automatic alarm convergence and self‑healing, and intelligent root‑cause engines that recommend remediation actions, moving the observability platform toward fully predictive, decision‑driven operations.