Tagged articles
parameter server
19 articles · Page 1 of 1
ComputeShare demonstrates how a centralized parameter‑server architecture, linear‑scaling learning‑rate adjustments, stale‑gradient protection, optional async SGD, gradient compression, and a universal dataset factory enable heterogeneous devices (CUDA, MPS, CPU) to collaboratively train models over LAN or the Internet, with benchmark results on MNIST, Fashion‑MNIST, and USPS.
ComputeSharePyTorchasync SGD
0 likes · 11 min read

The article details JD Retail's evolution from TensorFlow‑based sparse training to a custom high‑performance parameter server and a fully GPU‑accelerated, multi‑node, multi‑card synchronous training framework that leverages GPU‑RDMA, two‑level CPU‑DRAM/GPU‑HBM caching, and pipeline parallelism to overcome storage, I/O, and compute challenges of trillion‑parameter recommendation systems.
AI infrastructureGPU accelerationparameter server
0 likes · 12 min read

This article introduces NVIDIA's Merlin suite, covering product overview, Merlin Models & Systems, the TensorFlow Distributed Embedding (TFDE) plugin, the Hierarchical‑KV library, and the Hierarchical Parameter Server (HPS), while highlighting their architecture, performance benefits, and ease of integration for large‑scale recommendation workloads.
Distributed EmbeddingGPU accelerationHierarchical KV
0 likes · 13 min read

PetPS introduces a persistent‑memory‑based parameter server that redesigns indexing with the PetHash hash table and offloads parameter aggregation to NIC Gathering, achieving up to 1.7× higher throughput and significantly lower latency for industrial‑scale embedding models in recommendation, search, and advertising workloads.
Embedding Modelsparameter serverperformance optimization
0 likes · 14 min read


AntTechJul 12, 2023 · Artificial Intelligence
This article describes the Hybrid Embedding solution proposed by Ant AI Infra to address storage, resource, and feature‑governance challenges of massive sparse CTR models, detailing its multi‑layer storage design, KV‑based parameter server, and performance gains in large‑scale recommendation systems.
AI InfraCTRHybrid Embedding
0 likes · 9 min read

The award‑winning VLDB 2022 paper introduces HET, a cache‑enabled distributed framework that dramatically reduces communication overhead for sparse trillion‑parameter embedding models, and Tencent Ads has industrialized this technology to train 10 TB‑scale models with up to 7×24‑hour online deep learning.
CacheEmbeddingdeep learning
0 likes · 9 min read

The presentation by Tencent expert Yuan Yi details the Wuliang deep learning system for recommendation, covering its background, technical challenges such as massive data and real‑time requirements, the parameter‑server based solutions for training and inference, model compression techniques, and continuous online deployment strategies.
deep learninglarge-scale trainingparameter server
0 likes · 14 min read

This article presents an in‑depth overview of Tencent's Wuliang deep learning platform for recommendation systems, detailing the real‑time data challenges, high‑throughput requirements, parameter‑server architecture, model compression techniques, multi‑level caching, and answers to common technical questions.
Inference Servicedistributed trainingmodel compression
0 likes · 14 min read

The article introduces Angel Graph, a large‑scale graph computing platform built on Angel's parameter‑server architecture and Spark, detailing its evolution, framework components (including Spark‑on‑Angel and PyTorch‑on‑Angel), data and model partitioning strategies, communication and computation optimizations, stability mechanisms, usability features, and real‑world applications across recommendation, risk control, social and gaming domains.
PyTorchSparkgraph computing
0 likes · 15 min read

This article reviews how distributed machine learning tackles massive data and compute challenges by partitioning models and data across workers, optimizing communication with primitives, parameter servers, and Ring AllReduce, reducing IO overhead, and applying advanced optimizers such as LARS and LAMB to achieve faster, scalable training.
LAMB optimizerLARS optimizerRing AllReduce
0 likes · 31 min read

This article introduces Tencent's self‑built Angel distributed machine‑learning platform, describes its architecture and deep‑learning extensions (Parameter Server and AllReduce), explains how it powers the advertising recommendation pipeline with models such as DSSM, VLAD and YOLO, and presents extensive training‑level optimizations that yield multi‑fold performance improvements.
AngelRecommendation SystemTencent
0 likes · 15 min read

Deepthought is iQIYI’s end‑to‑end machine‑learning platform that unifies distributed frameworks, decouples pipeline stages, integrates with Tongtian Tower, and offers visual drag‑and‑drop configuration, evolving from a fraud‑detection prototype to a generic system with real‑time inference, automated hyper‑parameter optimization, and support for large‑scale data across anti‑fraud, recommendation, and analytics workloads.
AI PlatformAutoMLData Engineering
0 likes · 13 min read

The article explains how the fourth paradigm's distributed machine learning framework GDBT tackles the massive data, high‑dimensional features, and real‑time requirements of modern recommendation systems by leveraging heterogeneous computing, parameter servers, RDMA networking, and optimized workloads.
GDBTRDMAdistributed ML
0 likes · 18 min read

This article provides a comprehensive introduction to modern distributed systems, covering recent research trends, practical technologies such as Paxos, Consistent Hashing, MapReduce, Spark, various storage and computing paradigms, and offers guidance for beginners on how to navigate the field.
MapReducePaxosStorage
0 likes · 18 min read

This article interprets Alibaba’s “Image Matters” paper, explaining how the Deep Image CTR Model (DICM) introduces user‑side visual preference modeling with image embeddings, why traditional Parameter Servers struggle with large image vectors, and how the Advanced Model Server (AMS) compresses embeddings to enable efficient distributed training.
Advanced Model ServerCTRImage Embedding
0 likes · 15 min read


Didi TechJun 8, 2018 · Artificial Intelligence
DiDi PS is a custom RDMA‑based parameter server that uses a ring topology and optimized ibverbs communication to dramatically accelerate distributed deep‑learning training, consistently outperforming OpenMPI, NCCL2, TensorFlow’s built‑in RDMA, and Horovod while providing more stable and scalable synchronization for massive data workloads.
AllreduceRDMATensorFlow
0 likes · 10 min read

This article describes TensorFlowRS, an Alibaba‑built extension of TensorFlow that tackles the massive compute and sparse‑feature challenges of search, advertising and recommendation by redesigning the parameter server, adding fail‑over, gradient‑compensation, online‑learning support, advanced training modes and visualisation, achieving up to 100× training speedup and improved model quality.
TensorFlowdistributed trainingonline learning
0 likes · 16 min read


21CTOAug 13, 2017 · Artificial Intelligence
This article surveys distributed machine‑learning platforms, classifies them into basic data‑flow, parameter‑server, and advanced data‑flow models, examines Spark, PMLS (Petuum), TensorFlow and MXNet, presents performance comparisons on EC2 instances, and discusses bottlenecks, fault tolerance, and future research directions.
SparkTensorFlowparameter server
0 likes · 12 min read

This article surveys design approaches for distributed machine learning platforms, classifies them into basic dataflow, parameter‑server, and advanced dataflow models, examines examples such as Spark, PMLS, TensorFlow and MXNet, and presents performance evaluations and future research directions.
SparkTensorFlowdistributed machine learning
0 likes · 10 min read
