Data Warehouse Construction and Data Governance Practices at Wing Payment

Introduction

The digital era demands robust data warehouses and effective data governance. Huang Luo, a core member of Wing Payment’s financial data warehouse and governance team, shares experiences on building a powerful data warehouse and implementing governance practices.

Data Governance Background

Early challenges included code redundancy, unstable task timeliness, severe metadata loss, high data security risks, and inconsistent data definitions, highlighting the need for comprehensive governance across architecture, security, and operations.

Data Governance Construction Content

Organizational Collaboration: Establish data governance, technical architecture, and implementation committees to coordinate cross‑departmental goals.

Platform Construction: Build a data development platform supporting offline and real‑time scheduling, quality monitoring, and a self‑service BI platform for ad‑hoc queries and visual development.

Data Application Governance: Enhance usability, reduce compute/storage costs, and accelerate queries.

Data Standards: Define naming conventions, metadata management, and data classification.

Data Security: Secure storage, transmission, and usage to meet compliance.

Enterprise‑Level Data Warehouse Construction

The construction follows a layered architecture: ODS, DWD, DWS, DWM, DM, and APP layers, each with specific responsibilities such as data ingestion, detail processing, aggregation, and application delivery.

Key steps include:

Research Phase: Identify business pain points, organizational data needs, product processes, and technical schemas.

Platform Support: Upgrade from Hive to Spark, implement self‑service BI, metadata management, and indicator management platforms.

Data Warehouse Layering: Define ODS, DWD, DWS, DM, and APP layers with clear data flow and responsibilities.

Dimension Modeling: Choose business processes, define granularity, confirm dimensions and facts, and apply naming standards.

Data Governance Effectiveness

In 2023, governance reduced platform resource consumption by 86%, saved nearly ten million yuan annually, accelerated report generation, and improved data security through classification, encryption, and download approval.

Future Planning

Build a data‑warehouse cockpit for unified monitoring and daily task alerts.

Develop an asset management system displaying platform health, scheduling, storage, and security metrics.

Optimize indicator management to reduce duplicate processing.

Expand data empowerment via tag management, FTP distribution, and data interfaces.

These initiatives aim to enhance data accessibility, security, and business value across the organization.