JD Retail Data Asset Management: Upgrading Practices and Technical Innovations

JD Retail Data Asset Management – Overview

In the digital economy, data assets have become a core competitive advantage. JD Retail, as a leading e‑commerce platform, has been actively exploring and optimizing data asset management to improve operational efficiency and user experience.

Background and Challenges

Retail data models exceed 800,000 tables, including many temporary and invalid tables, making model discovery and usage difficult for analysts. The challenges include high data volume, complex dimensions, and the need to improve data retrieval efficiency, reduce storage‑compute costs, and provide trustworthy data for rapid business decisions.

Key Technical Innovations

Multi‑Level Acceleration Engines : RBO and HBO engines accelerate queries based on cost and usage scenarios, dynamically allocating storage‑compute resources for optimal performance.

Intelligent Materialization : Instead of the common cube pre‑compute + cache pattern, JD implements active metadata‑driven definitions and feedback‑based dynamic decisions, ensuring the most optimal materialization strategy per query.

Unified DSL for Querying : A JSON‑like domain‑specific language abstracts natural‑language requests into five elements – indicators, attributes, criteria, orders, and pagination. Example:

{
    "indicators": ["ge_deal_standard_deal_ord_amt"],
    "attributes": ["shop"],
    "criteria": {
        "criterions": [
            {"propertyName": "main_brand", "values": "8557", "type": "string", "op": "="},
            {"propertyName": "dt", "value": "2023-12-21", "type": "string", "op": "="}
        ],
        "orders": [{"ascending": false, "propertyName": "ge_deal_standard_deal_ord_amt"}],
        "maxResults": 5,
        "firstResult": 0,
        "group": ["shop"]
    }
}

Semantic Layer & Knowledge Graph : A data‑knowledge system standardizes asset definitions, security models, and consumption management, enabling end‑to‑end traceability and automated production.

Data Governance & Security : Role‑based data permissions, asset certification, and automated lifecycle management ensure data consistency, privacy, and compliance.

Data Asset Chapter – Asset Certification & Governance

Nearly 3,000 models have been certified with an 84% coverage rate across core business domains (transactions, users, traffic, marketing, finance). Asset perception is enhanced through automated graph construction, enriched model detail pages, and a standardized field library.

Data Capability Chapter – Indicator Middle‑Platform Practice

The middle‑platform provides full‑stack indicator management: asset control, topology, rule engine, anomaly detection, and intelligent acceleration. It supports real‑time monitoring, automatic scaling, and multi‑level caching (JIMDB + local cache) to improve TP99 performance.

Data Visualization Chapter – Visualization Tools

JD’s JMT visualization leverages advanced graphic grammar (DATA, TRANS, SCALE, COORD, ELEMENT, GUIDE) to build flexible components such as DuPont analysis, anomaly grids, and cross‑analysis tables. Low‑code orchestration enables complex page layouts, component configuration, and code generation.

Data Intelligence Chapter – Large‑Model Applications

LLM‑driven chatBI combines business knowledge with data assets via prompt engineering, entity extraction, and intelligent materialization. A local fine‑tuned model reduces latency and protects privacy while providing natural‑language query, descriptive analysis, and exploratory insights.

Business Impact

• Daily data calls exceed 40 million, supporting 8,000+ indicators and 22 data products. • End‑to‑end delivery time reduced from 3 days to 0.8 days (≈70% efficiency gain). • Low‑code + data push enabled 7,220 report emails during a major promotion. • Mobile low‑code apps delivered rapid “golden‑eye” analytics for field users.

Future work focuses on AI‑enhanced visual analysis, automated reporting, and continuous optimization of storage‑compute costs.