Data Development Production Environment Isolation: Xiaomi's Experience, Technical Choices, and Implementation

The presentation introduces the background of data development production environment isolation at Xiaomi, describing three stages of the data platform evolution: a multi‑platform phase, a unified data platform phase (with unified metadata, permissions, and scheduling), and an online development phase that includes environment isolation, standardized processes, and a WebIDE.

It outlines the challenges of data security, quality, and efficiency, emphasizing the need for regional, role‑based, and privacy data isolation to comply with domestic and international regulations.

Technical considerations compare physical isolation (separate test and production clusters) with logical isolation (single cluster with separate logical databases). Physical isolation offers stronger security but reduces flexibility, while logical isolation simplifies development but may not suit real‑time workloads. Xiaomi evaluates both and adopts a hybrid approach using naming environment variables to resolve database names per environment.

The productization path includes defining job states (development vs. production), introducing reviewers, automating code review, and implementing smart checks such as SQL static analysis, lineage verification, and self‑test enforcement. An MVP‑driven rollout across five phases gradually introduces version comparison, multi‑version workflows, environment‑aware naming, and approval integration.

Operational improvements focus on interactive development feedback by decoupling the scheduler from trial runs, adding an execution layer that provides low‑latency status and log events, and ultimately evolving toward a WebIDE experience.

Business practice results show that over 90% of teams adopted environment isolation, with a significant reduction in data‑quality incidents and streamlined approval processes. The Q&A discusses Git integration challenges and scenarios where physical isolation is preferred.