JD Retail Metric Middle Platform: Architecture, Semantic Layer, Production, Governance and Practical Cases

The presentation introduces JD Retail’s metric middle‑platform, built on modern data stack, headless BI, data virtualization and data weaving, aiming to provide an end‑to‑end solution from metric definition to production and consumption, and to promote a strategic, organizational approach to data.

Four key problems of previous metric systems are identified: difficulty finding metrics, ambiguous definitions, fragmented development, and inconsistent usage. The proposed solution includes a clear metric catalog, precise definitions, strong production capabilities, headless BI for unified consumption, and comprehensive governance.

The overall architecture separates user interaction (metric market, definition, semantic layer) from the data pipeline (production, acceleration) and metadata management, enabling systematic configuration and reduced manual effort.

The semantic layer is built independently, using the 4W1H framework (Who, What, When, Where, Why, How) to structure business language into atomic and derived metrics, with concepts such as modifiers, clipping calibers, and time/aggregation functions.

Metric production is configuration‑driven: users define physical tables, dimensions and modifiers, the system generates SQL, supports dimension joins, de‑duplication, secondary aggregation, and various acceleration strategies (media acceleration, lightweight aggregation, pre‑computation, intelligent acceleration) across engines like ClickHouse, HBase and Redis.

Acceleration techniques include query merging, predicate push‑down, attribute concatenation, and custom bucket depth to balance precision and performance, especially for high‑traffic scenarios like leaderboards.

Governance covers end‑to‑end monitoring: metric usage, quality scoring, recommendation of 4W1H elements, duplicate detection, lifecycle management of intermediate results, dynamic scaling, and low‑frequency data retirement, aiming for clear definitions, fresh data, efficient systems and cost‑effective operations.

Results after two years show the platform becoming the unified export for retail metrics, serving multiple products with daily calls exceeding 40 million, over 10 k registered metrics, covering more than 55 % of demand (80 % for new demand), and reducing manpower by over 50 %.

Future plans focus on broader metric usage, more compact data handling, and increased reliability, including stability improvements, semantic‑layer quality enhancements and leveraging large models to simplify metric discovery and registration.