Why Modern Enterprises Need a Data Middle Platform: Lessons from NetEase Yanxuan

Dear friends, the technical team of NetEase Yanxuan will share practical insights weekly, hoping we can improve together.

This article interprets the concept of a data middle platform based on NetEase Yanxuan’s experience building one.

Why are companies building data middle platforms?

Since 2018, discussions about data middle platforms have surged as many enterprises seek digital transformation and fine‑grained operations. Economic slowdown pushed companies to rely on data‑driven, precise management, requiring large‑scale data applications such as search, recommendation, BI reports, and data dashboards.

These needs demand robust underlying big‑data support, prompting the construction of data middle platforms when the scale of data applications justifies the investment.

Digital transformation requires multiple touchpoints—public accounts, mini‑programs, apps—and consequently needs search, recommendation, CRM, DMP, and risk control, all powered by big data.

Fine‑grained operations demand higher frequency and granularity of analysis, real‑time dashboards, and scenario‑based data products, all relying on a solid data foundation.

What is a data middle platform?

Definition: A data middle platform is a combination of high‑quality, efficient data systems and services that empower the data front‑end.

Its core responsibility is to efficiently enable the data front‑end (search, recommendation, BI reports, dashboards) to deliver business value.

What does a data middle platform contain?

It comprises the data warehouse system, a set of data services, and a BI platform.

Data Warehouse System

The data warehouse is the core of the platform, handling storage and computation independent of business logic. It includes management systems that ensure data completeness, consistency, and accuracy.

Key components:

Kuafu – event‑tracking management system ensuring data quality.

Jingwei – data entry system for importing and validating data.

Cangjie – metric management system defining and governing indicators.

Suiren – metric map for business users to view metric definitions.

Beidou – warehouse design management system that standardizes design before development.

Data Quality Center – provides end‑to‑end monitoring, impact assessment, and recovery.

Data Service Set

Data services encapsulate solutions for various data scenarios, accelerating application development. Core services include unified query service and tag service, which deliver metric and tag models to downstream systems.

The unified query service acts as a gateway, providing table‑to‑API conversion, data routing, rate limiting, and circuit breaking.

BI Platform

NetEase Yanxuan uses the "Yanxuan YouShu" BI platform, a customized version of NetEase YouShu. It offers an agile, PPT‑like interface for building reports, with caching strategies that achieve near‑100% first‑visit cache hit rates and overall hit rates above 80%.

Conclusion

When enterprises pursue digital transformation and fine‑grained operations that generate large‑scale data application demands, building a data middle platform becomes essential. A data middle platform is a high‑quality, efficient combination of systems and services that empower the data front‑end, encompassing a data warehouse, data services, and a BI platform.