Federated Learning with Differential Privacy: Algorithms and Performance Analysis

Federated Learning with Differential Privacy: Algorithms and Performance Analysis

8 Nov 2019 | Kang Wei, Jun Li, Ming Ding, Chuan Ma, Howard H. Yang, Farokhi Farhad, Shi Jin, Tony Q. S. Quek, H. Vincent Poor
This paper addresses the challenge of information leakage in federated learning (FL) by proposing a novel framework called Noising Before Model Aggregation FL (NbAFL). NbAFL adds artificial noises to the parameters at the client side before aggregation to prevent information leakage. The authors prove that NbAFL satisfies differential privacy (DP) under different protection levels by adjusting the variance of the added noises. They develop a theoretical convergence bound for the loss function of the trained FL model in NbAFL, revealing three key properties: 1) There is a tradeoff between convergence performance and privacy protection levels; 2) Increasing the number of clients participating in FL improves convergence performance given a fixed privacy protection level; 3) There is an optimal number of maximum aggregation times for a given protection level. Additionally, they propose a $K$-random scheduling strategy, where $K$ clients are randomly selected from the total $N$ clients for each aggregation, and develop the corresponding convergence bound. Theoretical results show that this strategy retains the above three properties and there is an optimal $K$ that achieves the best convergence performance at a fixed privacy level. Simulations validate the theoretical findings, demonstrating the effectiveness of the proposed methods.This paper addresses the challenge of information leakage in federated learning (FL) by proposing a novel framework called Noising Before Model Aggregation FL (NbAFL). NbAFL adds artificial noises to the parameters at the client side before aggregation to prevent information leakage. The authors prove that NbAFL satisfies differential privacy (DP) under different protection levels by adjusting the variance of the added noises. They develop a theoretical convergence bound for the loss function of the trained FL model in NbAFL, revealing three key properties: 1) There is a tradeoff between convergence performance and privacy protection levels; 2) Increasing the number of clients participating in FL improves convergence performance given a fixed privacy protection level; 3) There is an optimal number of maximum aggregation times for a given protection level. Additionally, they propose a $K$-random scheduling strategy, where $K$ clients are randomly selected from the total $N$ clients for each aggregation, and develop the corresponding convergence bound. Theoretical results show that this strategy retains the above three properties and there is an optimal $K$ that achieves the best convergence performance at a fixed privacy level. Simulations validate the theoretical findings, demonstrating the effectiveness of the proposed methods.
Reach us at info@study.space