Practical Engineering Guide to Federated Learning: Deployment, Training, and Inference

In recent years, federated learning (FL) has moved from a hot research topic to practical adoption, requiring a fast transition from theory to production. FL is a distributed machine‑learning paradigm that exchanges model parameters instead of raw data, thereby preserving data privacy while enabling collaborative model training across heterogeneous devices.

Deployment : Because enterprise environments differ in hardware and network conditions, the most convenient way to package an FL application is to bundle it and its dependencies into a lightweight Docker container. This container can run on any machine with Docker installed, similar to a shipping container that can be loaded onto various transport vehicles.

Significant time and resource savings (seconds‑level startup, MB‑level image size).

Eliminates environment‑setup issues for developers; Docker images provide a consistent runtime.

Facilitates continuous integration and service‑oriented architecture.

Supports multi‑platform deployment and standardized release processes.

Training Service Architecture : The training system consists of several key services:

Communication Service : Exposes gRPC/HTTP interfaces via a gateway, handling request routing and service discovery.

Training Service : Includes validation, task scheduling, metadata management, and federated‑learning components. Validation checks configuration correctness; the scheduler parses parameters and creates a responsibility‑chain of components; the metadata center records progress, status, and participant information; the FL component performs the actual model updates.

Model Management Service : Persists trained models, handles versioning and grouping.

Registry (e.g., ZooKeeper) : Registers service instances for high availability and load‑balanced routing.

The typical training workflow follows these steps (see 图二): submit training task → gateway routes to training service → validate parameters → load sample data → intersect feature IDs across parties → run federated training (e.g., LR, DNN) → evaluate model (AUC, KS, etc.) → store model metadata and artifacts.

Inference Service Architecture : After training, the model is deployed for inference, which demands low latency and high reliability. The inference stack mirrors the training stack with added emphasis on real‑time performance:

Communication Service: Proxy exposing gRPC/HTTP, optionally fronted by Nginx for load balancing.

Inference Service: Registers endpoints in ZooKeeper, pulls models from distributed storage to local cache, preprocesses features, and executes predictions.

Model Management Service: Stores persisted models and version information.

Storage Service: Caches prediction results and model files for fast recovery.

The inference workflow (see 图四) includes: submit inference task → gateway routes to inference service → fetch model (from cache or remote storage) → preprocess features on both parties → run joint prediction → return results via communication service → post‑process and store final outcomes.

Overall, the article demonstrates how to engineer a complete federated learning system—from containerized deployment to robust training and inference pipelines—while balancing flexibility, scalability, and operational stability.