Taodaji Cuts Troubleshooting 50% with Cloud-Native Observability Stack
Taodaji migrated from self-built SkyWalking and Elasticsearch to Alibaba Cloud's managed ARMS, SLS, Cloud Monitor 2.0, and STAROps, achieving 40% lower API latency pre-launch, minute-level fault localization, 50% faster troubleshooting, and 30% operational cost reduction through unified traces, logs, metrics, and AI-driven root cause analysis.
Hubei Taodaji Supply Chain operates the B2B agricultural trading platform "Youdaji," where order concurrency is high and transaction chains are long. Any system stall or data inconsistency directly translates into delivery delays and inventory mismatches, making stability a business lifeline rather than a technical nice-to-have.
Background: The Build-vs-Buy Dilemma
As the new core system adopted a microservice architecture with deep call chains and high expected concurrency, the team initially planned to use a self-built SkyWalking + Elasticsearch (ES) stack. While workable at small scale, the exponential growth of Trace, Metric, and Log data exposed three structural problems that could not be solved by simply adding machines.
Three Core Challenges
Self-built cluster maintenance burden and stability risk: Continuous ES index optimization, shard/replica tuning, JVM tuning, and version upgrades consumed core R&D resources. Under peak loads, massive trace and log writes risked ES cluster instability — exactly when observability was needed most.
Data silos and slow end-to-end diagnosis: Traces, metrics, and logs lived in separate systems without a unified correlation key. During pre-launch stress testing, a slow core interface required engineers to manually jump across three UIs, three query syntaxes, and stitch timestamps and TraceIDs by hand. Average root-cause localization exceeded 30 minutes.
Alert noise and lack of sustainable capability: Static threshold alerts generated storms during traffic spikes or network jitter, burying the true root cause. The ad-hoc troubleshooting practices built for launch could not be retained as daily operational assets.
Solution: Four-Layer Cloud-Native Observability Platform
After comparing the self-built route with Alibaba Cloud's managed services, Taodaji chose ARMS + SLS + Cloud Monitor 2.0 + STAROps for two reasons: zero-maintenance managed services with smooth migration, and native integration of traces, metrics, logs, topology, and AI diagnosis.
(1) Managed Tracing & Smooth Migration: ARMS Replaces SkyWalking+ES
ARMS Application Monitoring ingests data via OpenTelemetry standard protocol and Java Agent bytecode instrumentation — zero business code changes. Existing SkyWalking instrumentation habits and trace views carried over seamlessly. ARMS supports asynchronous context propagation across thread pools and message queues, keeping TraceIds intact across complex calls. The managed storage and query engine eliminates ES index tuning, shard management, JVM tuning, and stability firefighting, guaranteeing second-level write and query stability under high concurrency.
(2) Unified View Breaking Data Silos: Cloud Monitor 2.0
Business logs are centralized into SLS Log Service with TraceIDs embedded as correlation keys. Cloud Monitor 2.0 unifies metrics, logs, and traces in a single platform, creating a closed observability data loop. Engineers can now jump from an abnormal span in the trace topology directly to the corresponding business logs in SLS at the exact timestamp, and vice versa. The previous 30-minute manual stitching process becomes a single TraceID-driven automatic correlation of "trace view + metric curve + log details."
(3) Dynamic Topology Visualization: UModel
UModel automatically builds a real-time full-chain topology — frontend → backend → middleware → database → containers — without manual diagramming. The topology refreshes automatically on deployments, dependency changes, or container scaling, turning a static outdated diagram into a live map with real-time health status. Clicking any node reveals its associated metrics, traces, and logs, providing a unified visual entry for capacity planning, bottleneck prediction, daily inspections, and blast-radius analysis.
(4) AI-Driven Diagnosis: STAROps
STAROps layers AI on top of Cloud Monitor 2.0's unified data (Metric, Log, Trace, Event). It performs automated root-cause reasoning by traversing the UModel topology backward along dependency edges, pinpointing the source node and propagation path amid alert storms. It also executes proactive inspections to detect risk precursors before they become incidents, shifting operations from reactive firefighting to proactive prevention. Alert noise reduction and topology-aware correlation let the team move from "manual trace digging" to "AI proposes root cause, human confirms."
Measurable Outcomes
Pre-Launch Optimization: 40% Latency Reduction
Using ARMS full-chain tracing down to method-level code lines and slow SQL statements, the team eliminated performance bottlenecks on core trading paths before launch. Core interface average response time dropped ~40%, ensuring smooth high-concurrency go-live.
Fault Localization: From >30 Minutes to Minutes
Unified view + TraceID-native linkage + STAROps topology-aware AI root cause analysis cut average fault localization time from over 30 minutes to minutes. Overall troubleshooting efficiency improved ~50%, MTTR drastically reduced. For a fresh-produce supply chain, faster diagnosis directly means fewer delivery delays and inventory mismatches.
Cost & Asset Sustainability: ~30% Ops Cost Reduction
Managed platform eliminated self-built cluster hardware, endless tuning, and dedicated firefighting headcount. Pre-launch performance baselines, alert rules, and capacity models solidified into daily operational capabilities. STAROps AI root cause analysis and proactive inspections transformed the observability stack from a one-off project expense into a reusable, growing long-term asset. IT infrastructure and personnel ops costs fell ~30%.
Future Roadmap
Full business-line coverage: Extend the same managed observability and intelligent diagnosis to all lines — fresh procurement, warehousing/logistics, processing, multi-level distribution — establishing a single operational standard and data dialect.
From post-mortem to pre-emption: Combine UModel dynamic topology, performance baselines, and STAROps proactive inspection with capacity forecasting and trend prediction to intervene before risks materialize (e.g., pre-scale services ahead of promotions).
From technical observability to business metrics: Align technical stability indicators with business continuity measures — order fulfillment timeliness, transaction success rate, inventory sync accuracy — so technical decisions carry a business-value yardstick.
Related Products: ARMS Application Monitoring (https://www.aliyun.com/product/arms), SLS Log Service (https://www.aliyun.com/product/sls), Cloud Monitor 2.0 (https://www.aliyun.com/product/cms), STAROps Intelligent Ops Assistant (https://www.aliyun.com/product/starops).
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
