How JD Retail Overcomes Data Governance Challenges to Boost Efficiency

JD Retail faces multiple data management and governance challenges: continuous data growth creates inefficient and redundant data models, raising maintenance costs and harming data quality; shared account resources lack proper change management, leading to operational risks; and expanding table counts and storage scales increase compute and storage consumption.

Key Topics Covered

Data Management Challenges

Data Governance System Construction

Proactive Metadata Governance Practice

Summary and Future Outlook

Q&A

1. Data Management Challenges

Weak Asset Awareness : Difficulty locating assets among hundreds of thousands of data models, many temporary, invalid, or duplicate tables, leading to low confidence in data usage.

Non‑Agile Data Architecture : Coupled dimensions and pre‑computations, extensive wide tables causing high resource consumption and long delivery cycles.

Development Quality and Security Issues : Uncontrolled table structure changes, operational risks from parameter mismatches, and developers writing directly to production tables.

Rising IT Resource Costs : Continuous growth in table numbers and storage, low utilization of invalid or similar tables, and increasing compute costs.

2. Data Governance System Construction

(1) Standard Governance

JD Retail established a unified data language standard defining model elements such as business domain, subject, process, attributes, update frequency, and granularity. Using this standard, high‑quality, high‑value models are certified and cataloged, while low‑quality models are decommissioned to free resources.

The standardization also systematizes dimension and metric registration, enabling automatic collection of table metadata for downstream modeling, intelligent table inspection, and automated production.

(2) Architecture Governance

Architecture is made more agile by adopting logical virtual tables that model data as dimensions and metrics, reducing physical wide‑table dependencies. Intelligent materialization (HBO, CBO, RBO) automatically decides which paths to pre‑materialize, cutting manual effort and IT costs.

JD Retail also explores lake‑warehouse integration, leveraging incremental state updates and batch‑stream convergence to improve processing efficiency and lower data costs.

(3) Development Governance

Development‑production isolation separates accounts, tables, and queues, ensuring secure data production.

(4) Resource Governance

Resource governance includes storage lifecycle management (identifying and retiring invalid or similar tables, compression, redistribution) and compute governance (detecting idle tasks, optimizing low‑utilization jobs, peak‑shaving, and engine tuning).

Proactive metadata mining builds governance models and visual dashboards, enabling data owners to see resource distribution, governance outcomes, and pending issues.

Conclusion

By establishing comprehensive standards, agile architecture, isolated development, and systematic resource management, JD Retail achieves a sustainable, efficient, and secure data governance ecosystem that supports rapid business growth.