Tagged articles
TensorFlow Serving
10 articles · Page 1 of 1
The iQIYI deep‑learning platform identified two TensorFlow Serving memory‑leak problems—a string‑accumulating executor map caused by unordered input maps and an uncontrolled gRPC thread surge under heavy load—submitted upstream patches that sort inputs and cap thread counts, eliminating OOM crashes and stabilizing production.
Memory LeakTensorFlow ServinggRPC
0 likes · 10 min read

By adding model warm‑up files, separating load/unload threads, switching to the Jemalloc allocator, and isolating TensorFlow’s parameter memory from RPC request buffers, iQIYI’s engineers reduced TensorFlow Serving hot‑update latency spikes in high‑throughput CTR recommendation services from over 120 ms to about 2 ms, eliminating jitter.
Model Hot UpdateTensorFlow Servingjemalloc
0 likes · 11 min read

This guide explains the core concepts of TensorFlow Serving, shows how to prepare Docker images, save TensorFlow 2.x models in various formats, configure version policies, warm‑up models, start the service, and invoke it via gRPC or HTTP with complete code examples.
DockerHTTPTensorFlow Serving
0 likes · 11 min read

This article explains the core concepts of TensorFlow Serving, shows how to prepare the environment with Docker, convert TensorFlow 2.x models to the SavedModel format, configure version policies, warm‑up the service, and invoke predictions via gRPC or HTTP interfaces.
DockerHTTPPython
0 likes · 11 min read

This article details how Jike's recommendation team upgraded from Spark to TensorFlow, introduced a twin‑tower embedding model for recall, deployed it with TensorFlow Serving and Elasticsearch, and achieved a 33.75% lift in user interaction on the dynamic square.
ElasticsearchEmbeddingRecommendation System
0 likes · 9 min read

This article explains how to prepare the Docker environment, install TensorFlow Serving (CPU and GPU versions), convert a YOLO V3 checkpoint to SavedModel, deploy the model as a service, warm‑up and manage versions, invoke it via gRPC and HTTP, and compare CPU versus GPU inference performance.
AIDockerGPU
0 likes · 9 min read

Meituan’s delivery ETA system progressed from linear regression to DeepFM, enriching user, rider, merchant, and spatiotemporal features, employing an asymmetric loss and business‑rule integration to favor early arrivals, adding a tail‑adjustment term, and is engineered with Spark‑assembled TFRecords, multi‑GPU TensorFlow training, and remote‑served TensorFlow Java inference achieving sub‑5 ms TP99 latency.
ETALong TailTensorFlow Serving
0 likes · 15 min read


58 TechNov 21, 2018 · Artificial Intelligence
This article describes the architecture, components, and deployment strategies of the 58 deep learning online prediction service, covering TensorFlow‑Serving, custom model serving, traffic forwarding, load balancing, GPU configuration, resource monitoring, and the supporting web management platform.
GPUTensorFlow Servingload-balancing
0 likes · 15 min read

Meituan’s user‑growth team built a Wide‑Deep CTR prediction model, trained offline with Spark‑generated TFRecords, and deployed it via TensorFlow Serving on YARN, then applied request‑side multithreading, offline one‑hot preprocessing, XLA JIT compilation, and dedicated loading threads to cut end‑to‑end latency from ~18 ms to ~6 ms and eliminate model‑switch spikes.
TensorFlow Servingdistributed trainingmodel deployment
0 likes · 15 min read

This article provides a step‑by‑step tutorial on deploying trained deep‑learning models to production, covering client‑server architecture, load balancing with Nginx, using Gunicorn and Flask, cloud platform choices, autoscaling, CI/CD pipelines, and additional tools such as TensorFlow Serving and Docker.
APIDockerTensorFlow Serving
0 likes · 11 min read
