Building a Closed‑Loop AI System: From Data Collection to Model Deployment in Alibaba’s XiaoMi

With the rise of AI, increasingly complex algorithmic products require a rapid 0‑to‑1 model construction and continuous iteration; this article describes how to close the loop of data analysis → sample annotation → model training → monitoring feedback to support such systems.

Background : XiaoMi provides intelligent services for consumers and merchants, handling billions of dialogues during events like Double‑11. It uses various algorithms (Q&A, recommendation, decision) and faces challenges in quickly building and iterating models.

Implementation – Phase 0→1 (Cold‑start) : Extract dialogue logs, perform knowledge mining, annotate data, train models, evaluate jointly, and release the model.

Implementation – Phase 1→100 (Bad‑case feedback) : Collect bad‑case information from user feedback, analyze data, retrain affected models, and redeploy.

Pain points : Different algorithms need varied annotation interfaces; annotation quality lacks guidance; bad‑case detection and repair are slow; maintaining hundreds of models consumes engineering effort; data samples moving between business and algorithm teams pose security risks.

Closed‑loop model architecture : Consists of four layers—dialogue system, data layer, sample layer, and model layer—forming a continuous training loop.

Data layer : Multi‑dimensional data queries and OLAP data cubes are used; challenges include high‑cardinality dimensions and complex OR queries.

Sample layer : Annotation components support various labeling formats; high‑dimensional data is visualized using PCA/T‑SNE, vectorized with word2vec or pHash, and clustered with k‑means.

Visualization and interaction : Dimensionality reduction produces scatter‑plot collapses, enabling class‑level inspection and keyword extraction via box‑selection.

Real‑time defense : Logs are streamed through Flink for real‑time clustering; high‑frequency issues are displayed in an annotation UI for rapid keyword addition and model update.

Data processing engine : Built on Spark, with client and compute‑cluster components; supports local and remote execution, SQL and MLlib libraries, and integration into templated workflows.

The presentation concludes with a summary of benefits and a thank‑you note.