How Federated Learning Balances Privacy and Collaboration in AI

Abstract

Federated Learning (FL) is a core paradigm of privacy‑preserving machine learning (PPML) that balances collaborative computation with data protection. By allowing participants to exchange only model parameters or gradients, FL breaks data silos, enables cross‑institution data value release, and builds multi‑layer defenses through differential privacy, homomorphic encryption, and secure multi‑party computation.

1. Basic Introduction

1.1 Background

AI applications in healthcare, finance, recommendation, etc., generate highly distributed heterogeneous data. Traditional centralized training requires aggregating all data, which faces storage, compute, and strict privacy/compliance constraints, leading to data islands. Federated Learning emerges to enable cross‑institution collaborative modeling without raw data sharing.

1.2 What is Federated Learning

Federated Learning is a distributed machine‑learning framework that keeps local data on each participant while jointly optimizing a global model through local training and a central or multi‑party aggregation mechanism. FL consists of three core mechanisms:

1) Local training mechanism : Participants train models on their own data, keeping raw data local.

2) Encrypted parameter transmission : Model updates are protected with homomorphic encryption, differential privacy, etc., preventing leakage during communication.

3) Global aggregation mechanism : A trusted server or secure multi‑party computation aggregates encrypted updates to produce an improved global model.

Compared with other PPML approaches, FL offers stronger robustness in heterogeneous data environments by “exchanging models instead of data”.

2. Technical Solutions

2.1 Definition and Development

FL allows many users to collaboratively train a global model without sharing their raw data. If we denote the accuracy of a centralized model as and that of a federated model as , the existence of a non‑negative satisfying indicates an accuracy loss of the federated algorithm.

2.2 Technical Classification

Based on the distribution of features, labels, and sample IDs among participants, FL can be divided into three categories:

Horizontal Federated Learning (HFL) : Participants share the same feature space but have different sample spaces (e.g., banks in different regions). HFL focuses on joint modeling across samples.

Vertical Federated Learning (VFL) : Participants share the same sample space but have different feature spaces (e.g., a bank and a supermarket serving the same customers). VFL enables joint modeling of different attributes of the same users.

Federated Transfer Learning (FTL) : Participants have little or no overlap in both feature and sample spaces (e.g., a Chinese bank and a US e‑commerce platform). FTL introduces transfer learning to share knowledge via a small common dataset.

2.3 Technical Development

Practical FL faces challenges such as model‑training attacks, communication overhead, data availability, device instability, and fairness. Consequently, FL techniques are organized into four major directions:

Communication : Local multiple‑epoch training, model/gradient compression, structured/quantized communication, edge‑cloud hierarchical collaboration, and decentralized P2P architectures (e.g., FedAvg).

Privacy Protection : Secure multi‑party computation (MPC), homomorphic encryption (HE), differential privacy (DP), secure aggregation (SecAgg) to keep data “usable but invisible”.

Heterogeneity : Knowledge distillation, model compression, personalized/multi‑task learning, hierarchical scheduling, and elastic computation to handle diverse devices and data distributions.

Incentive Mechanisms : Contribution measurement (Shapley value), game‑theoretic resource allocation, crowdsourcing, blockchain, and smart contracts to ensure fair participation.

2.4 Horizontal FL Implementation (FedAvg)

FedAvg is the classic HFL framework. Its workflow includes:

System initialization – define global model architecture, optimizer, training rounds, local epochs, batch size, and optional security settings.

Client sampling – coordinator selects a subset of participants.

Model broadcast – global parameters are sent to selected clients.

Local training – each client runs several epochs on local data, optionally applying gradient clipping and DP noise.

Secure aggregation – only aggregated sums are revealed to the coordinator.

Model update – weighted averaging of client updates (FedAvg).

Convergence check – stop if accuracy/loss/round criteria are met; otherwise repeat.

2.5 Vertical FL Implementation

Vertical FL typically consists of sample alignment and joint training phases. Sample alignment matches records across parties while keeping raw data local. Joint training uses encrypted model parameters; a trusted third party may manage keys, decryption, and aggregation, or the label‑holding party can act as coordinator.

System initialization – third party generates a public/private key pair (e.g., Paillier) and distributes the public key.

Model structure agreement – participants decide how to split the model (bottom layers on one side, top layers on the other) and initialize parameters.

Training parameter setting – define rounds, iterations per round, batch size, gradient clipping, DP usage, etc.

Per‑round process – participants compute forward passes on encrypted features, exchange encrypted embeddings, compute loss, encrypt gradients, and send them to the third party for decryption.

Third‑party decryption – recovers gradients and loss, forwards them back to participants.

Model update – participants remove masks, apply gradient descent, and check convergence.

2.6 SDH Implementation Case

Alibaba’s Secure Data Hub (SDH) adopts the Elastic‑Federated‑Learning‑Solution (EFLS). EFLS uses differential privacy for gradient protection by splitting each batch into micro‑batches, clipping gradients, and adding Gaussian noise. It also employs homomorphic encryption for privacy‑preserving forward propagation, enabling encrypted feature exchange and secure model updates in two‑party scenarios.

3. Application Scenarios

Smart city traffic: multiple cities jointly train traffic‑flow prediction models without sharing raw travel data.

Intelligent risk control: banks and e‑commerce platforms collaboratively train anti‑fraud models via VFL.

Advertising conversion prediction: ad platforms and advertisers combine online behavior and offline conversion data through VFL.

Autonomous driving decision optimization: transfer learning adapts models trained on normal roads to extreme weather or remote areas.

4. Summary

Federated Learning shifts the focus from moving data to adapting models to data, allowing training to stay on local nodes while a coordinator aggregates model updates. By transmitting only encrypted gradients or parameters, FL fundamentally prevents data leakage, satisfies regulatory constraints, and enables scalable, privacy‑preserving AI across domains.

5. References

[1] McMahan H B, Moore E, Ramage D, et al. Communication‑efficient learning of deep networks from decentralized data. ICML 2017.

[2] Yang Q, Liu Y, Chen T, et al. Federated machine learning: Concept and applications. ACM TIST 2019.

[3] Truex S, Baracaldo N, Anwar A, et al. A hybrid approach to privacy‑preserving federated learning. ACM AI‑Security 2019.

[4] Zhao Y, Li M, Lai L, et al. Federated learning with non‑iid data. arXiv 2018.

[5] Li T, Sahu A K, Talwalkar A, et al. Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine 2020.

[6] Liu L, Zhang J, Song S H, et al. Client‑edge‑cloud hierarchical federated learning. ICC 2020.

[7] Khan L U, Pandey S R, Tran N H, et al. Federated learning for edge networks: Resource optimization and incentive mechanism. IEEE Communications Magazine 2020.

[8] Cong M, Yu H, Weng X, et al. A game‑theoretic framework for incentive mechanism design in federated learning. Springer 2020.