Mastering Gray-Scale Cross-Platform Monitoring for Front-End Apps

Hello, I am Mo Hui from Alibaba Taobao Technology Department. Today I will share the topic "How to Build Gray-Scale Cross-Platform Monitoring".

Background

Monitoring is the front line of safe production; the ability to discover online problems quickly and locate them is the core capability of monitoring. Front‑end cross‑platform solutions are constantly evolving, covering Web, Weex, mini‑programs, and other scenarios. The discussion is organized into four dimensions: background, technical solution, online case, and summary.

From a traditional front‑end monitoring perspective, we usually focus on runtime exceptions such as JS errors, interface failures, and performance metrics. However, our business runs on many cross‑platform solutions (Weex, mini‑programs, RN, Flutter, etc.), requiring full‑link monitoring for issues like container start failures, white‑screen loads, and crashes.

Analysis of Taobao front‑end incidents shows that 80% of online problems are caused by changes, with an average repair time exceeding 300 minutes. The main reasons are insufficient log growth visibility and lack of fine‑grained dimensions to distinguish log sources.

Overall log growth is not obvious.

Missing detailed dimensions to identify the source of log increase.

The fault‑repair process consists of two stages: development (requirements review, development, testing) and release (gradual rollout from internal gray to external gray, then full launch). Discovering problems only after full launch leads to long repair cycles and wide impact, which can be mitigated by early detection during gray‑release.

Technical Solution

The overall solution is divided into four parts:

Gray‑release: two types – zip‑package deployment for mini‑programs and static resource deployment to CDN for regular pages.

Log collection: capture metrics such as JS errors and interface exceptions during page execution.

Data processing: clean and normalize the reported data.

Gray‑monitoring: build monitoring based on the processed data.

For static‑resource releases (Web, Weex), gray control is performed at the CDN layer. If the request hits the gray version, the gray resources are served; otherwise, the online version is served.

Zip‑package releases have two scenarios:

Mini‑programs: packaged as zip and deployed.

Web offline: static resources are bundled into an offline zip for deployment.

Both zip and non‑zip releases distinguish current and online versions using three fields:

Version number.

Gray‑hit flag.

Environment identifier.

External (Web) collection uses a global variable injected during the build process, e.g.:

<meta name="page-tag" content="env=spe,grey=true,release=0.0.1" />

Container collection reads response headers, e.g.:

grey: 'true'
env: 'spe'
release: '0.0.1'
date: 'Fri, 11 Dec 2020 13:12:04 GMT'
content-type: 'text/html; charset=utf-8'
...

Data processing on the TT streaming platform includes:

Acquiring raw logs from various containers.

Real‑time cleaning and formatting using Blink.

Storing cleaned logs in SLS for real‑time query and analysis.

Node‑level applications for real‑time log queries and alerting.

By adding version, environment, and gray‑hit dimensions to logs, we can differentiate new error logs and calculate growth ratios, enabling gray‑alerting and real‑time gray monitoring.

Gray‑alerting runs as a Node process that polls page release lists, compares online and gray version logs, and triggers alerts based on new log counts and growth ratios.

Gray‑real‑time monitoring visualizes core metrics such as gray traffic proportion and detailed log status (e.g., JS error similarity analysis) on dashboards.

Online Case

In a real deployment, a new feature rollout with ~5% gray traffic showed a 5‑fold increase in error logs, prompting an immediate rollback that prevented a larger outage.

Summary

Gray‑scale monitoring consists of four processes:

Gray‑release: covering Web, mini‑programs, Weex, etc.

Metric collection: browser variables for Web, response headers for containers.

Monitoring metrics: full‑link coverage with logs carrying gray version info and standardized format.

Gray applications: using gray dimensions for log analysis, gray‑alerting, and real‑time monitoring.