JD Daojia Machine Learning Platform: Architecture and Implementation

This article introduces JD Daojia's machine learning platform, which was developed to meet the growing demand for advanced algorithm capabilities in their instant retail business. The platform consists of three main subsystems: model training, feature model management, and online model prediction services.

The model training platform uses Kubernetes for distributed resource management and TensorFlow 2.0 for deep learning model training. It supports both Multi-Worker and Parameter Server distributed training methods to handle large-scale data and complex model structures. The platform provides features like sample experiments, model experiments, resource management, debug tools, and real-time A/B testing.

The feature model management platform addresses the need for managing multiple business scenarios and models. It provides model file management, basic feature management, and model feature management capabilities. The platform supports hot deployment and replacement of online models, reducing manual intervention and improving operational efficiency.

The online model prediction service provides unified API interfaces for model prediction requests and abstracts complex feature data queries and model update logic. It implements scenario-based isolation deployment to handle different business scenarios independently, improving system stability and resource planning.

The platform has been successfully applied in various business scenarios including homepage feeds recommendation, channel feeds recommendation, and global search. In the homepage feeds recommendation scenario, the platform supports multi-objective deep expert network structures that combine embedding processing, feature engineering, and transformer-based sequence encoding to achieve a 30%+ improvement in click-through rate.

The article also discusses future directions including graph technology exploration using Graph Attention Networks (GAT) to address user interest expression and cold start problems. The platform continues to evolve to support new algorithm technologies and accelerate business development across JD Daojia's operations.