How JD Cloud’s Federated Learning Platform Breaks Data Silos for AI

Overview

As online businesses generate massive data, many organizations face data silos caused by competition, privacy concerns, and complex administrative procedures, making cross‑department or cross‑company data integration costly or impossible.

At the same time, growing emphasis on data privacy and security poses new challenges for artificial intelligence, requiring frameworks that can train models collaboratively without exposing raw data.

Federated learning, first proposed by Google in 2016, is an emerging AI technology that enables multiple participants to train machine learning models jointly while keeping data locally, protecting privacy, and ensuring compliance. It supports algorithms beyond neural networks, such as random forests, and is expected to become a foundation for next‑generation collaborative AI.

JD Cloud Federated Learning Platform

The JD Cloud Federated Learning platform was created to provide a distributed‑data‑set based federated learning solution. During training, model updates are exchanged in encrypted form, preventing any party from seeing another’s raw data, while the final model can be shared across institutions.

Architecture

The platform consists of a federated learning client and a JD Cloud gateway. The client handles data encryption and scientific computation, while the gateway transmits encrypted parameters between participants.

Security Features

The gateway performs system authentication and only allows participants to send requests to it; the gateway cannot initiate requests to participants, enabling a one‑way network transmission that protects participants’ inbound traffic.

The platform supports two sample‑alignment methods—federated encrypted alignment using RSA with random noise, and MD5 alignment—ensuring that shared user IDs are matched without revealing non‑shared IDs.

Supported Algorithms and Tools

The platform offers Logistic Regression, XGBoost, and Deep Neural Networks, along with feature analysis methods such as Pearson, Spearman, WOE, and IV. It also provides preprocessing utilities like outlier filling, normalization, binning, Count Encoding, and One‑Hot encoding.

Implementation Details

The platform is built on TensorFlow 2.0 and its high‑level tf.keras API, avoiding dependence on Spark, Yarn, or Kubernetes. It uses the Subclassing API for flexible model definition, allowing a single forward pass during training, which reduces runtime and eliminates instability caused by random numbers.

Deployment and Use Cases

Deployment can be completed within three days, with the platform ready for use after one week. It supports visual feature analysis without coding.

The platform has been applied in retail, automotive, education, and risk control. In the automotive sector, federated training improved model performance by 17% within two weeks, boosting conversion rates and ROI.

One automotive brand integrated online and offline data from multiple 4S stores via the platform, enabling accurate predictions of store visits and model preferences, and significantly increasing sales efficiency.