How Douyin’s Data Asset Platform Redefines Big Data Lineage

Overview

The Douyin Group Data Asset Management Platform provides a unified portal for managing, discovering, and utilizing data assets across complex business scenarios, shifting the focus from traditional metadata to a systematic "manage‑find‑use" approach.

Platform Features

The platform ingests metadata from diverse data sources into a centralized metadata lake, including full‑link lineage. Data business partners perform secondary management such as publishing, classification, and asset evaluation, while AI‑driven search and recommendation capabilities enable efficient consumption of data assets.

Full‑Link Data Lineage

The presentation concentrates on the platform’s full‑link lineage, covering four main topics: an overall introduction, system architecture, application scenarios, and future outlook.

Construction Background

Viewing the entire data chain helps understand relationships among millions of tasks.

Ensuring quality by assessing the impact of frequent online task changes.

Maintaining security by efficiently detecting sensitive data propagation.

Reducing costs by identifying low‑value resources for governance.

Future Outlook

Building robust big‑data lineage is deemed urgent to support quality, security, and cost‑effective data governance across the organization.