How Alibaba’s Big Data Model Governance Boosted Efficiency and Cut Costs

Data Situation

Alibaba’s large‑scale data system has achieved a leading position, but rapid growth in data volume and developers exposed problems such as non‑standard table naming, low reuse of common‑layer tables, excessive temporary tables, uneven table ownership, and complex cross‑market dependencies, leading to higher costs and lower efficiency.

Problem Analysis

The issues span evaluation, construction, management, and usage phases. Evaluation lacks a unified assessment framework; construction suffers from missing end‑to‑end modeling tools; management is hindered by high costs, slow iteration, and uneven ownership; usage faces difficulty in finding and trusting data.

Solution

Key goals include model digitization, public model sinking, productization of a full‑cycle modeling tool, daily governance, and search‑enhancement for data retrieval.

DataWorks Co‑development

DataWorks provides a one‑stop big‑data development and governance platform built on MaxCompute/EMR/Hologres. Joint development added intelligent modeling, data mapping, and development assistants, enabling reverse and forward modeling, visual design, Excel and code‑based modeling, and automatic ETL code generation.

Model Scoring

A digital dashboard evaluates models at project, owner, and BU levels, offering governance suggestions and leveraging lineage and tagging for precise interventions.

Find‑Data Efficiency

Improvements include enhanced search filters, upgraded table description editors, collaborative data albums, and a data‑knowledge chatbot to streamline table discovery and usage.

Summary and Future Plans

The initiative delivered a model evaluation system, intelligent modeling capabilities, upgraded data maps, and clear governance policies. Future work will focus on unifying architecture and standards, strengthening the common layer, controlling ADS complexity, improving governance enforcement, and further integrating modeling with data maps.