Design and Practice of Turing OS: An Online Service Framework for Machine Learning and Deep Learning at Meituan

The article introduces the Turing platform built by Meituan Delivery's technology team, a one‑stop algorithm platform that provides end‑to‑end services from data preprocessing, feature generation, model training, evaluation, deployment, online prediction, A/B testing, and algorithm effect assessment. The online service framework, Turing OS, focuses on machine‑learning and deep‑learning online services, offering a unified solution for deploying models and algorithm strategies at scale.

Background : Early in Meituan Delivery, each business line independently developed online prediction components (the “chimney model”), leading to duplicated effort, lack of platform capabilities, and tight coupling between algorithms and engineering. As the business grew, these issues became bottlenecks, prompting the creation of a unified online service framework.

Turing OS 1.0 : Integrated model computation with preprocessing and post‑processing via an SDK. It solved wheel‑reinvention, simplified development, and reduced coupling, but still suffered from high coupling among algorithm packages, business services, and the platform, as well as deployment‑related pain points.

Turing OS 2.0 addresses the shortcomings of 1.0 with several key features:

Standardized lightweight SDK : A minimal SDK that hides routing, feature fetching, and model execution, lowering integration effort.

Algorithm pluginization : Algorithms are packaged as plugins and hot‑deployed in Turing OS via custom class loaders, enabling version‑independent releases.

Data channel : Allows algorithms to fetch required data autonomously, reducing the need for business services to mediate data flow.

Algorithm orchestration (DAG) : Operators (model‑compute or algorithm‑compute) can be composed serially or in parallel, forming directed acyclic graphs for complex workflows.

Multi‑mode integration : Supports Standalone (service‑level deployment) and Embedded (in‑process deployment) modes, each with trade‑offs in coupling and performance.

Turing sandbox : An isolated environment mirroring production for safe pre‑release validation, traffic diversion, and performance stress testing.

Unified replay platform : Records inputs, outputs, features, and models using Elasticsearch and HBase, enabling offline replay and debugging.

Performance testing & optimization : Leverages Meituan’s Quake full‑link testing system to generate traffic from replay data, perform stress tests, and produce diagnostic reports.

Results : Over 20 Turing OS clusters have been deployed, supporting 25+ algorithm packages, 50+ algorithms, and processing billions of online predictions daily. Algorithm iteration cycles have been reduced from days to hours, with most steps fully automated and independent of engineering or testing teams.

Future outlook : Plans include automated operation and testing tools, deeper integration with Spark for large‑scale validation, and a graphical workflow engine to further accelerate algorithm development and deployment.