How to Build a GPU‑Accelerated Distributed ML Platform for VM Migration Prediction

Background

In cloud environments, virtual‑machine (VM) hot migration demands minimal downtime; predicting future VM workload helps identify optimal migration windows.

Solution

We built a GPU‑accelerated distributed machine‑learning platform called XiaoZhuGe on Alibaba Cloud to predict ECS VM load and migration downtime. The platform integrates a web service, message queue, Redis, data acquisition (SLS, MaxCompute, HybridDB), OSS model repository, GPU cloud servers, Dask distributed framework, and the RAPIDS acceleration library.

The overall architecture is illustrated below:

The front‑end provides a Tengine+Flask web service for receiving computation requests, while a message queue decouples it from the back‑end compute cluster.

Dask manages data preparation, model training, and prediction tasks across GPU workers. MaxCompute handles offline training data, Blink processes real‑time data, HybridDB (Cstore) stores aggregated data for low‑latency online prediction, and OSS stores training data and models.

Design Considerations

Message queue (RocketMQ) decouples front‑end and back‑end, enabling high concurrency, fault tolerance, and horizontal scaling.

GPU‑accelerated parallel computing uses Dask with RAPIDS, achieving significant speedup over CPU‑only clusters.

Data platform combines ODPS, SLS, and Cstore to handle massive real‑time and batch data, with aggregation to avoid overload.

Framework choice: Dask was selected over Spark due to its lightweight nature and native RAPIDS integration.

GPU Acceleration

RAPIDS (Real‑time Acceleration Platform for Integrated Data Science) provides GPU acceleration for data‑science and machine‑learning workloads. Using RAPIDS, we reduced the required infrastructure from over 50 large CPU servers to just 8 small GPU servers, cutting costs to about one‑tenth.

Model Update and Evaluation

The platform provides automated online real‑time prediction; model evaluation, update, and release are being further automated to achieve a fully end‑to‑end workflow.

Conclusion

Under a cloud‑native paradigm, leveraging public‑cloud services simplifies building production‑grade platforms. XiaoZhuGe demonstrates the practical benefits of GPU acceleration for large‑scale machine‑learning tasks, delivering significant performance gains and cost reductions.