This article details the architecture and practical techniques behind WeChat's large‑scale monitoring system, covering lightweight data collection, classification of real‑time, non‑real‑time and user‑specific metrics, anomaly detection algorithms, automated configuration, and high‑performance storage solutions for billions of events per minute.
At Meitu’s seventh Internet Technology Salon in Xiamen, over a hundred experts discussed recommendation algorithms and big‑data solutions, with talks on the Arachnia log‑collection system, the Naix distributed bitmap service, Meitu’s personalized recommendation pipeline challenges, and novel data‑missing‑theory models for improved performance.
The article details WMDA, a no‑code and manual‑code data collection platform for PC, mobile and app that supports real‑time and offline user behavior analysis, describing its functional model, behavior taxonomy, five‑layer architecture, tracking techniques, circle‑selection, data services, streaming and batch processing pipelines, and related technologies such as Storm, Spark, Druid and Roaring Bitmap.
The article explains how to move beyond macro app metrics by collecting offline and real‑time user data, storing it in HDFS, processing it with Spark, visualizing behavior paths as state‑machine trees, and performing branch‑funnel analysis to uncover conversion bottlenecks and improve product quality.
This tutorial walks through using Python and BeautifulSoup to crawl user profiles, fan lists, and song information from the Quanmin K‑Song app, clean and store the data in MongoDB, handle pagination, and prepare the dataset for further analysis.
This article details Qunar's hardware automation operations platform, covering the hardware scope, pain points of manual processes, a five‑stage lifecycle, automated testing, data collection, fault handling, and the underlying Mesos‑Marathon‑Docker infrastructure that together improve efficiency, reliability, and cost control.
This article explains how to design and implement a comprehensive operations monitoring platform by integrating Ganglia for data collection and Centreon for alerting, detailing a six‑layer architecture, data flow, seamless integration, and practical Q&A for real‑world deployment.
This article details a step‑by‑step process for crawling all popular Ximalaya channels, extracting each audio's metadata, and storing roughly 700,000 records in MongoDB, while also showing how to speed up the crawl with asynchronous requests.
This article details the design, technology selection, architecture, encryption, compression, and performance evaluation of Ctrip's real-time user data collection system, which leverages Java, Netty, Kafka, and Avro to achieve high throughput, low latency, and robust fault tolerance for mobile and web applications.
By defining data collection (event tracking) requirements in a structured JSON format and storing them in Git with a web interface that abstracts version control, teams can standardize identifiers, validate data formats automatically, track changes via commit logs, and streamline collaboration between product managers, developers, and testers.
This article explains what user feedback is, why it should be analyzed, and provides a step‑by‑step methodology—including channel setup, data collection, coding, categorization, and statistical analysis—along with key performance indicators for monitoring feedback handling in product management.