Data Tracking and Analysis System (Part 1): Architecture, Data Collection, and Visualization

Background

As the company’s business and platform complexity grow, user experience and operational efficiency are increasingly challenged. To pinpoint user pain points, improve conversion, and empower business, a data‑driven optimization strategy is essential.

The ZooTeam front‑end team launched an internal product called the "Hunyin System" at the end of last year to provide web data‑tracking and quantitative analysis capabilities, and shares lessons for teams building similar solutions from scratch.

System Overview

The data‑tracking analysis system consists of four parts that together form a complete user‑behavior analysis solution:

Tracking Collection JSSDK : collects user actions and reports them.

Data Processing Service : receives, stores, filters, processes, and exposes the data.

Data Visualization Platform : aggregates and displays detailed data with customization.

Chrome Extension Tool : visually presents hotspot data on the page for a more friendly experience.

When a user visits any instrumented page, actions such as page entry, clicks, and scrolls are classified by the JSSDK, reported to the backend, queried by the site or extension, processed, and finally rendered on the visualization platform.

Data Collection

Data collection can be categorized into three approaches:

Zero‑code (full) tracking : captures all user actions without any development effort, resulting in large volumes and noise.

Visual tracking : non‑developers select tracking points directly on the page.

Code‑intrusive tracking : developers embed tracking code into business logic, offering higher accuracy at higher cost.

Because high accuracy is required while keeping early development effort low, the team primarily adopts the code‑intrusive tracking method, supplemented by an automated injection mechanism that reduces integration cost. Two sending methods are used: automatic DOM‑attribute‑based sending and manual custom sending.

// Automatic sending example
<button data-utm-click="${did}" data-utm-data="${businessData}">

const utmCnt = g_UTM.batchSend('triggerType', [
  { utmCD:['blockInfo','positionInfo'], bdata:{key:'otherBusinessData'} },
  { utmCD:['001','008'], bdata:{key:'value'} }
]);

The collected fields focus on two themes: Event (type, timestamp, page position, etc.) and User (IP, OS, browser, screen resolution, etc.), with a unique user identifier stored in a cookie.

{
  bdata: {},
  createTime: "1571038815128",
  evt: "browse",
  ipAddr: "122.226.174.195",
  logType: 2,
  lver: "1.1.0",
  mx: 0, my: 0,
  os: "Windows/7",
  pre: "https://www.zcygov.cn/",
  scr: "1920x1360",
  url: "https://www.zcygov.cn/",
  userId: "001",
  utmCnt: "a0004.2ef5001f.0001.0001.d814bf60ee5511e99397b37fe9083257",
  utmUrl: "a0004.2ef5001f.0001.0001",
  uuid: "d7fd8de0-ee55-11e9-9397-b37fe9083257"
}

Data Display

The front‑end portal provides rich visualizations, including PV/UV trends, funnel analysis, path analysis, heatmaps, user portraits, custom dashboards, and various business‑specific reports and exports.

PV/UV trend and ranking

Funnel analysis for conversion rates

Path analysis for source and destination pages

Click heatmaps (scroll heatmaps coming soon)

User portrait (upcoming)

Customizable dashboards

Business Enablement

To ensure the collected data is valuable, the team benchmarked mature solutions such as GrowingIO, SensorsData, and Shujike, and identified key functional modules needed for their own system.

Feature

Shujike

SensorsData

GrowingIO

Self‑built

Form analysis

Supported

Supported

Supported

Not supported

Page analysis

Supported

Supported

Supported

Supported

Path analysis

Supported

Supported

Supported

Supported

Funnel analysis

Supported

Supported

Supported

Supported

Event analysis

Supported

Supported

Supported

Supported

Event distribution

Supported

Supported

Supported

Supported

User segmentation

Supported

Supported

Supported

Supported

Behavior prediction

Not supported

Supported

Not supported

Not supported

User behavior sequence

Not supported

Supported

Not supported

Not supported

Heatmap

Supported

Supported

Supported

Supported

Video replay

Supported

Not supported

Not supported

Not supported

Most of these functions are already available in the "Data Display" module; form analysis and user‑behavior sequence are planned for future releases. Form analysis helps identify drop‑off points in multi‑step forms, while behavior sequences allow per‑user path inspection.

Search‑Flow Tracking Example

The article illustrates a simple e‑commerce search flow: entering the homepage, typing a keyword, clicking search, viewing results, and clicking a product. Three types of data are collected on the three pages:

Automatic page‑enter/leave tracking

Button‑click tracking

Link‑click tracking

From these logs we derive PV, UV, button click counts, and link click counts (the latter identified via the originating button’s unique code). Further processing yields stay time, conversion rate, and heatmaps.

Summary

This post provides an overview of a generic data‑tracking and analysis system, outlining its core capabilities and future improvement directions. Subsequent articles will dive into specific challenges, optimization techniques, and detailed implementations.

How to Efficiently and Completely Collect Data

Designing a Tracking Data Analysis Model

Chrome Extension for Data Visualization