Mastering Data Governance: From Metadata to ETL in One Guide

1. Data Governance Framework – Mind Map

2. Metadata

2.1 Issues Addressed by Metadata

What data – what is data – where does it come from – how does it flow – who can access

Its essence is also a kind of data; managing metadata is the foundation of data‑asset management.

2.2 Metadata Classification

Business metadata : describes concepts, relationships and rules related to business domains, such as business terms, information categories, metrics, statistical definitions.

Technical metadata : describes technical concepts, relationships and rules, including definitions of platform objects, data structures, source‑to‑target mappings, and transformation processes.

Management metadata : describes management‑related concepts, relationships and rules, mainly personnel roles, responsibilities and management processes.

2.3 Metadata Model Maturity

1. Stage 1 – manual metadata management, requiring extra steps outside the governance process.

2. Stage 2 – automatic metadata generation during data discovery.

3. Stage 3 – automatic construction of data‑flow metadata.

2.4 Metadata Construction Goals and Management Measures

2.5 Metadata Management

Metadata management methods:

Metadata management capabilities:

More reference: First data‑middle‑platform metadata standard (download)

3. Data Standards

Main components :

3.2.2 Types of Data Standards (Examples)

Different industries have different standards; examples include codes for gender, ID number, amount, phone number, industry, level classifications, etc.

3.3 Data Standard Management System Approach

Data standards originate from business and serve business.

Build based on existing standards

Basic data standards: business‑oriented view.

Indicator data standards: management‑oriented view.

Data standard specification is a business‑driven, external‑requirement‑based, enterprise‑situation‑based compatible process.

3.4 Data Standard Architecture System

Not all basic data need standards; data items must meet sharing, importance and feasibility criteria.

3.5 Principles for Management Data Standard Construction

Definition : analysis‑type data standards must align with business meaning and applicable scenarios.

Scope : business value range, calculation method and coding rules must stay consistent.

Name : Chinese and English names follow unified naming rules; identical business meaning items keep the same name.

Reference : external standards (international, national, industry) and internal policies must be consistent.

Source : each standard must have an authoritative source system; other systems should directly use the authoritative result.

3.6 Data Standard Lifecycle Management

4. Data Modeling

4.1 Concept

Enterprise‑level data model construction method : start from a global perspective, standardize data models, build a unified control system, enrich entity attributes, clarify logical relationships, and form domain‑specific models.

4.2 Data Model Classification

4.3 Data Model Lifecycle

4.4 Case Study

5. Data Integration

5.1 Concept

Data integration refers to the process of centrally managing business data from disparate information systems, continuously integrating new and different data sources to provide a foundation for data sharing.

5.2 Overall Architecture of Data Integration

6. Data Lifecycle

6.1 Phase Division

Business planning definition phase: business planning, business standard design

Application design implementation phase: data model design, application standard design, application development, data entry

Data lifecycle management phases :

Data creation: ensure completeness via data model, accuracy via standards, quality checks, and proper system generation.

Data usage: monitor usage with metadata, ensure accuracy with standards and quality checks, control derivation.

Data archiving: evaluate timing and archive by data type.

Data destruction: evaluate timing and destroy by data type.

Requirements :

Meet policies and management rules for historical data queries.

Support business operations and analytical needs.

Fulfill audit management requirements.

Reduce data redundancy and improve consistency.

Invest in storage, hardware, and operations infrastructure.

Enhance application performance and response speed.

6.2 Management Requirements and Measures

6.3 Management Norms and Methods

7. Data Quality

7.1 Data Quality Management Goals

Develop management methods that meet data‑consumer quality requirements.

Define quality control standards and embed them throughout the data lifecycle.

Establish processes to measure, monitor, and report quality levels.

Identify and promote improvement opportunities by adjusting processes, systems, and activities based on consumer needs.

7.2 Lifecycle

Planning stage : quality team assesses problem scope, impact, priority, and solution alternatives.

Execution plan : team addresses root causes and plans continuous monitoring.

Check stage : actively monitor data quality against requirements.

Handle stage : resolve newly emerging quality issues.

7.3 Data Quality Dimensions

Detailed dimensions are described in another document.

7.4 Common Data Quality Tools

8. Data Development

Design the full‑process data development management around the data value chain (data asset → data service → business application) to unlock data value.

8.1 Data Asset

8.2 Data Service

Architecture:

9. Data Security

10. ETL

10.1 Definition

10.2 ETL Modes

Trigger mode

Incremental field mode

Full sync mode

Log comparison mode

Comparison of modes

10.3 Offline and Real‑time

Usage scenarios: