Tagged articles
distributed machine learning
17 articles · Page 1 of 1
Federated Learning enables multiple parties to collaboratively train a global AI model without sharing raw data, using techniques like local training, encrypted parameter exchange, and secure aggregation, while addressing privacy, communication efficiency, heterogeneity, and incentive challenges across horizontal, vertical, and transfer learning scenarios.
Federated LearningHorizontal FLSecure Aggregation
0 likes · 24 min read


Didi TechJan 25, 2024 · Artificial Intelligence
Didi’s new Ray‑native XGBoost training platform replaces the fault‑prone Spark solution with a fully Pythonic, fault‑tolerant architecture that leverages Ray’s autoscaling and gang‑scheduling, delivering 2–6× speedups, reduced failure rates, efficient sparse‑vector handling, scalable hyper‑parameter search, and improved resource utilization for large‑scale machine‑learning workloads.
MLOpsRayXGBoost
0 likes · 20 min read

Meituan’s Fulfillment Platform team built a high‑performance distributed causal‑forest framework—named Causal On Spark—that trains hundreds of trees on hundreds of millions of samples within minutes using MapReduce‑based histogram splitting, extensive memory optimizations, Parquet model serving, and novel distributed evaluation metrics, enabling scalable causal inference for pricing, subsidies, and marketing.
Causal InferenceModel ServingSpark
0 likes · 23 min read

This article reviews how distributed machine learning tackles massive data and compute challenges by partitioning models and data across workers, optimizing communication with primitives, parameter servers, and Ring AllReduce, reducing IO overhead, and applying advanced optimizers such as LARS and LAMB to achieve faster, scalable training.
LAMB optimizerLARS optimizerRing AllReduce
0 likes · 31 min read

This article introduces JD's open‑source federated learning platform 9N‑FL, explaining the data‑island problem, the fundamentals and classifications of federated learning, its four key features, the system’s layered architecture, development timeline, real‑world advertising use case results, and future enhancements.
9N-FLFederated LearningJD
0 likes · 15 min read


58 TechNov 20, 2020 · Artificial Intelligence
The article details the development, architecture, and optimization of 58.com’s AI algorithm platform (WPAI), covering its background, overall design, large‑scale distributed machine learning, deep‑learning platform features, inference performance enhancements, GPU resource scheduling improvements, and future directions.
AI PlatformGPU schedulingInference Optimization
0 likes · 15 min read

This article introduces Tencent's self‑built Angel distributed machine‑learning platform, describes its architecture and deep‑learning extensions (Parameter Server and AllReduce), explains how it powers the advertising recommendation pipeline with models such as DSSM, VLAD and YOLO, and presents extensive training‑level optimizations that yield multi‑fold performance improvements.
AngelRecommendation SystemTencent
0 likes · 15 min read

This article explains federated learning as a privacy‑preserving distributed machine learning paradigm, discusses why it has become popular, describes its three core components, demonstrates its advantages over traditional models, outlines real‑world use cases in medicine and finance, and analyzes current technical and commercial challenges together with implementation techniques such as horizontal/vertical federation and homomorphic encryption.
data securitydistributed machine learning
0 likes · 32 min read

This article examines federated learning’s wide‑range application prospects in healthcare, mobile internet, and finance, analyzes the technical and regulatory challenges of deploying such systems, and explains the concrete implementation steps for horizontal and vertical federated learning architectures.
AIFederated Learningdata security
0 likes · 11 min read

This article explains the rapid rise of federated learning, its technical foundations combining machine learning, distributed computing, and privacy protection, practical use cases, intuitive privacy examples, and empirical evidence that it can improve model performance without compromising data security.
Federated LearningModel Performanceartificial intelligence
0 likes · 15 min read

Alibaba’s eXtreme Parameter Server (XPS) platform tackles the challenges of training models on billions of samples and trillions of features by employing streaming learning, feature hashing, dynamic sparsity, asynchronous checkpointing, and high‑performance communication, enabling efficient, fault‑tolerant distributed AI at massive scale.
Alibabadistributed machine learningfeature hashing
0 likes · 21 min read

Kunpeng is a unique distributed platform that seamlessly integrates large‑scale system architecture with parallel optimization algorithms, delivering fault‑tolerant, high‑performance machine‑learning capabilities for billions of samples and features, and outperforming Spark, MPI, and XGBoost in real‑world Alibaba applications.
artificial intelligencedistributed machine learningscalable systems
0 likes · 11 min read

This article surveys design approaches for distributed machine learning platforms, classifies them into basic dataflow, parameter‑server, and advanced dataflow models, examines examples such as Spark, PMLS, TensorFlow and MXNet, and presents performance evaluations and future research directions.
SparkTensorFlowdistributed machine learning
0 likes · 10 min read

The article discusses Microsoft’s research on scalable distributed machine‑learning on Azure, highlighting the challenges of communication overhead, the use of Vowpal Wabbit and Statistical Query Model techniques, and proposing algorithms that reduce iteration counts to achieve faster, cost‑effective predictive analytics for large‑scale data.
Azure MLcommunication bottleneckdistributed machine learning
0 likes · 12 min read


21CTOSep 19, 2015 · Artificial Intelligence
The article explains how distributed machine learning evolved from parallel computing to handle massive, long‑tail data sets, discusses the importance of scalability, fault recovery, and data‑parallel algorithms, and reviews frameworks such as MPI, MapReduce, and Pregel for building large‑scale AI systems.
LDAMPIMapReduce
0 likes · 24 min read


21CTOAug 21, 2015 · Artificial Intelligence
This article explains how Facebook tackles massive recommendation data—over 100 billion ratings—by using distributed collaborative filtering, matrix factorization, SGD/ALS hybrid algorithms, and a novel work‑to‑work communication scheme built on Apache Giraph to achieve high performance and scalability.
ALSApache GiraphFacebook
0 likes · 9 min read

The article explains how Facebook built a large‑scale distributed recommendation system using Apache Giraph, collaborative filtering with matrix factorization, SGD and ALS algorithms, a novel work‑to‑work communication scheme, and performance optimizations that achieve ten‑fold speedups on billions of ratings.
ALSApache GiraphFacebook
0 likes · 9 min read
