Federated Learning and the 9NFL Platform: Architecture, Features, and Real‑World Applications

Data is the cornerstone of artificial intelligence, but increasing privacy regulations such as GDPR and China’s Data Security Management Measures make cross‑organization raw data sharing difficult, creating data islands that hinder insight generation.

Federated Learning (FL), first proposed by Google in 2016, provides a machine‑learning framework that enables multiple parties to collaboratively train models while preserving user privacy, complying with data‑security and regulatory requirements.

The 9NFL (Jiǔshù Federated Learning) platform was designed by JD.com’s Commercial Promotion Division in late 2019 and launched within six months. Its three main innovations are:

Independent R&D of cutting‑edge technologies, delivering an end‑to‑end solution from sample matching to model training and prediction.

Application in marketing recommendation, supporting billions of samples and hundreds of terabytes of data, achieving the first online deployment in e‑commerce recommendation.

Evolution for complex scenarios, implementing distributed asynchronous frameworks, failover, and congestion‑control mechanisms to ensure high availability across domains and networks.

9NFL builds on JD’s 9N machine‑learning platform, whose core is TensorFlow optimized for performance and scheduled by Kubernetes (k8s). It adds multi‑task cross‑domain scheduling, high‑performance networking, large‑scale sample matching, cross‑domain joint training, and hierarchical model encryption.

The platform consists of five modules:

Overall scheduling module for parallel multi‑task control.

Resource management module using virtualization to maximize utilization and hide infrastructure differences.

Cross‑network transmission module with fault‑tolerant, high‑throughput protocols.

Sample matching module supporting massive multi‑modal cross‑domain matching.

Trainer module enabling large‑scale distributed joint modeling.

Key design principles include:

Componentization: Layered, modular design allowing flexible component composition for different scales and granularity.

High Availability: Redundant packet filtering, retransmission, timeout strategies, message‑queue aggregation, and failover mechanisms to keep training uninterrupted.

High Performance: Distributed horizontal scaling, asynchronous multi‑task execution, and network‑level optimizations such as small‑packet merging and jump‑packet acknowledgments.

In a real‑world advertising scenario, 9NFL combined JD’s commercial interest data with media‑side user interest tags via federated learning, improving ad recall, CTR, and CVR models and delivering more than a 10% revenue increase.

Future work focuses on balancing encryption overhead with model complexity, simplifying deployment, and enhancing the user interface. The team invites contributors to join the open‑source project (link provided) and help advance federated learning as a foundational AI technology.