Building and Managing a Metric System in Data Warehouse: Practices from Dongchedi
This article details how the Dongchedi business team designs, implements, and monitors a comprehensive metric system within its data warehouse, covering metric standards, model construction, metadata management, quality monitoring, application scenarios, and future directions using the DataLeap platform.
The article introduces the critical role of a metric system in data analysis and application, then shares how the Dongchedi business builds and operationalizes its metric system from a data‑warehouse development perspective.
It outlines the six main topics of the presentation: establishing metric standards, converging metric model construction, quality monitoring strategies, full‑stack application scenarios, future outlook, and a Q&A session.
Business background: Dongchedi is a one‑stop automotive information and service platform with massive data volume (hundreds of PB) and diverse C‑end and B‑end use cases, requiring high‑quality data services from the data warehouse.
DataLeap platform: Serves as a unified metadata hub for metric definitions, supporting standardized naming, modeling, and service capabilities, and powers downstream BI tools such as the Metric Observation Platform.
Metric system framework: Consists of three layers—basic capability construction (metadata standards, model mounting, lineage, service quality), metric service capability (routing, fault tolerance), and application capability (visual query, BI platforms, commercial services).
Metadata management standards cover naming conventions, business definitions, level management (four‑level hierarchy), cataloging, versioning, and business naming, with a root‑based naming approach (data domain, process, measure, modifier, time period) and automated tools for root extraction.
Metric model construction converges on four common issues (insufficient dimension tables, missing aggregation layer, chaotic detail layer, duplicated application layer) and proposes a layered architecture (DWD, DWA, DM) with clear responsibilities for each layer.
Quality monitoring strategy includes three pillars: metric specification monitoring, query service monitoring (consistency, slow/abnormal queries, SLA, data drift), and governance monitoring, supported by a visual monitoring dashboard.
Application scenarios span internal dashboards, self‑service BI, commercial operation platforms, and full‑link lineage management, enabling consistent metric usage across the organization.
Future outlook envisions integration with Balanced Scorecard, unified data service query layers, and large‑model‑driven natural language data interaction.
The Q&A section addresses storage handling of decomposed metrics, duplicate root prevention, cross‑domain data consistency, multi‑model binding, lineage analysis methods, and the distinction between dimensions and modifiers.
DataFunTalk
Dedicated to sharing and discussing big data and AI technology applications, aiming to empower a million data scientists. Regularly hosts live tech talks and curates articles on big data, recommendation/search algorithms, advertising algorithms, NLP, intelligent risk control, autonomous driving, and machine learning/deep learning.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.