Building and Managing an Indicator System in a Data Warehouse – Lessons from Dongchedi
This article explains how Dongchedi’s data‑warehouse team designed, implemented, and monitored a comprehensive indicator system, covering metric standards, model construction, metadata management, quality control, and diverse application scenarios to support both C‑end and B‑end business needs.
The article introduces the importance of an indicator system for data analysis and application, and outlines six key topics that will be covered: establishing indicator standards, convergence of indicator modeling in the warehouse, quality monitoring strategies, building comprehensive application scenarios, metadata management standards, and future outlook.
It describes Dongchedi’s business scope, including content, tools, and community services, and highlights the massive data‑warehouse scale (hundreds of PB, tens of thousands of daily tasks, over 6,000 active indicators) that supports both offline, near‑real‑time, and real‑time data services.
The DataLeap platform is presented as a unified metadata hub that defines, models, applies, and serves business indicators, enabling a unified view of metric definitions and facilitating downstream BI products such as the indicator observation platform.
The indicator system construction framework consists of three layers: basic capability (metadata standards, model mounting, lineage, service quality), indicator service capability, and application capability. Basic capabilities include naming conventions, model granularity, lineage management, and service quality assurance.
Metadata management standards cover naming, business definitions, level management (four‑level hierarchy), catalog, version, and business naming. A root‑word decomposition method (data domain, business process, measure, modifier, time period) is used to generate standardized English and Chinese metric names, with an online tool to automate the process.
Indicator model construction addresses common problems such as insufficient dimension tables, missing aggregation layers, chaotic detail layers, and duplicated application layers. The solution involves a layered architecture (detail DWD, lightweight aggregation DWA, data‑mart DM) and a data‑asset map to avoid redundant model development.
Quality monitoring is divided into three parts: indicator specification monitoring, query service monitoring (including multi‑model consistency, slow queries, SLA, data drift), and governance monitoring, supported by a visual monitoring platform that tracks access heat, alerts, and governance reviews.
Comprehensive application scenarios are built on stable indicator definitions, robust models, and monitoring, serving internal dashboards, self‑service BI, and external commercial platforms, with full‑link lineage management to ensure impact analysis and SLA governance.
The future outlook includes integrating the indicator system with Balanced Scorecard (BSC) for strategic management, expanding unified data‑service query layers, and exploring large‑model‑driven data‑intelligence interactions.
A Q&A section addresses technical details such as storage of decomposed indicators, handling of synonymous root words, multi‑model binding, and lineage analysis.
DataFunSummit
Official account of the DataFun community, dedicated to sharing big data and AI industry summit news and speaker talks, with regular downloadable resource packs.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.