How Small Big‑Data Frontend Teams Can Thrive: A Survival Guide

Big Data Overview

Big data provides two core values: raw data services (e.g., pandemic contact‑tracing) and data‑driven insights (e.g., e‑commerce recommendation, short‑video personalization). A typical end‑of‑year report measures success by tool replacement, project coverage, data volume reduction and load‑time improvement.

Typical End‑to‑End Workflow

Define the business problem and required outcomes.

Analyze requirements, extract key metrics (usually by data analysts and operations).

Identify data sources such as event‑tracking logs, transactional databases, or third‑party APIs.

Design data domains, atomic metrics and calculation logic (the core modeling step).

Implement data pipelines, run batch or streaming jobs, and validate the results.

Data Warehouse Architecture

The Alibaba‑style layered architecture separates data into four logical zones:

ODS (Operational Data Store) – stores raw, unprocessed source data exactly as it arrives from business systems.

DWD (Detail Data Warehouse) – transforms ODS data into wide, denormalized tables that preserve business‑process granularity.

DWS (Data Warehouse Service) – aggregates DWD tables into reusable, metric‑level views with unified definitions.

ADS (Application Data Service) – exposes business‑oriented data sets (e.g., dashboards, APIs) built on top of DWS.

Warehouse engineers abstract product requirements into reusable models and maintain a library of internal business schemas that can be shared across projects.

Data Platform Functions

The data platform provides a closed‑loop technical product that manages:

Batch job scheduling and execution.

Data integration, extraction, and back‑flow.

Data quality monitoring and compliance.

These capabilities relieve warehouse engineers from operational overhead and ensure that downstream business lines receive reliable, governed data.

Visualization Asset System (Underlying Capability)

A standardized visualization specification and plug‑in asset system enable reuse across projects.

Basic chart component library.

Business‑specific custom components.

3D rendering capabilities (e.g., d3, three.js).

Drag‑and‑drop interaction and workflow orchestration (e.g., G6).

Integration with scene building and asset management.

Challenges

Balancing strict visual constraints with creative freedom.

High configuration complexity; requires strong abstraction skills.

Solutions

Define a standardized specification that separates variable items from constrained items and combine it with theme packages for different visual tones.

Classify assets into categories such as material, information, controls, maps, charts, tables, and business components.

Provide a fixed development scaffold, templates, and asset‑management tooling to reduce setup time.

Reference implementations: “Lu Ban Data Source Management Solution” and “Low‑Code Platform Remote Component Loading Solution”.

Visualization Building Platform (General Service)

The platform adopts a No‑Code approach for most visualizations; only occasional data‑rule transformations require front‑end code.

Large‑screen dashboards.

BI dashboards (interactive, often embedded in management systems).

Report exports (Excel).

PPT export.

PDF export.

Challenges

Complexity of the builder exceeds that of ordinary page builders due to richer configuration options.

Need for a clear abstraction layer between assets and the builder, plus a complete asset system and co‑creation rules.

Identifying primary beneficiaries, ownership of maintenance, and cost allocation.

Defining the boundary between platform evolution and daily development work.

Solutions

Standardize specifications for asset management, rendering, drag‑and‑drop, and data conversion.

Supply industry‑specific template solutions that business units can select.

Clarify responsibilities for front‑end, product, and visual teams during training and hand‑over.

Support multiple delivery formats such as npm packages, standalone HTML, or URL endpoints.

Event‑Tracking Analysis Capability

Challenges

Establish a standardized end‑to‑end event‑tracking process from requirement gathering to front‑end implementation.

Clarify data flow, distinguish front‑end vs. back‑end tracking, and decide between real‑time and batch (T+1) processing.

Promote product adoption despite low data awareness among users.

Standardize, visualize, make tracking unobtrusive, and engineer it.

Solution

Build a horizontal capability that emphasizes engineering practices over technical difficulty, enabling teams to instrument events consistently and reuse the instrumentation layer across products.

Digital Operations via Data Middle‑Platform

Leverage existing big‑data capabilities to connect business and product teams, helping product teams discover technical leverage points.

Event tracking requires collaboration among product, operations, and technology; front‑end executes the instrumentation.

Product data analysis inevitably relies on a visualization platform.

After analysis, front‑end can deliver experience‑focused solutions or concrete battle‑ready features.

Further digital‑operation solutions will be detailed in subsequent articles.

For reference, see the following posts (URLs retained for technical context):

Lu Ban Data Source Management Solution – https://juejin.cn/post/7122240814108901406

Low‑Code Platform Remote Component Loading Solution – https://juejin.cn/post/7127440050937151525

Component Online Preview and Debug – https://juejin.cn/post/7145604963593355277

Event‑Tracking Capability Share – https://juejin.cn/post/7114450860335169543

If you need to contact the team for technical collaboration, use the following address:

[email protected]