Architecture Design of a Front-End Monitoring Platform

Platform Background The front‑end monitoring platform (http://jscout.58corp.com) collects, analyzes, and displays JavaScript exception information to fill the gap after deployment, providing data for FE engineers to locate and debug errors. It consists of three parts: JS SDK, data analyzer, and web display.

Components

JS SDK: simple configuration to integrate into pages, customizable per business line.

Data Analyzer: scheduled tasks scan and aggregate exception data, persisting statistics.

Web Display: users log in to view real‑time exception info, trends, and query specific entries.

Architecture Design The design draws from common log‑collection systems (e.g., Meituan, JD) and the BetterJS front‑end monitoring solution. It includes an Agent layer for log collection, an Acceptor server behind Nginx, Kafka for buffering high‑throughput logs, MongoDB for raw storage, MySQL for processed data, and Hive for offline aggregation. HBase is considered for high‑speed function execution.

Key design points include horizontal scalability at each layer, use of message queues to handle attacks and concurrency, partitioned MySQL tables for storage, and a hybrid MySQL‑Hive storage model to balance real‑time and batch processing.

Operational Considerations

Collector layer can be horizontally scaled.

Kafka buffers traffic and allows consumer scaling, tolerating occasional out‑of‑order or lost logs.

Storage uses MySQL sharding; HBase offers faster function execution but MySQL excels at CRUD operations.

Basic Conventions Data is partitioned by project ID, separate tables per business line, date‑based splitting for large tables, retention of at least one year with staged deletions, and quarterly/annual Hive jobs for statistics.

Conclusion The platform’s architecture is optimized for the identified performance bottlenecks, employing MySQL sharding with offline Hive tasks, and will evolve with ongoing operational feedback.