Personalized Federated Learning with Moreau Envelopes

Personalized Federated Learning with Moreau Envelopes

26 Jan 2022 | Canh T. Dinh, Nguyen H. Tran, Tuan Dung Nguyen
The paper introduces pFedMe, a personalized federated learning (FL) algorithm that addresses the challenge of statistical diversity among clients, which hinders the performance of global models. pFedMe uses Moreau envelopes as regularized loss functions to decouple personalized model optimization from global model learning. Theoretically, pFedMe achieves state-of-the-art convergence rates: quadratic speedup for strongly convex objectives and sublinear speedup of order 2/3 for smooth non-convex objectives. Experimentally, pFedMe outperforms vanilla FedAvg and Per-FedAvg in both convex and non-convex settings on real and synthetic datasets. The key contributions include the formulation of a bi-level optimization problem, leveraging the properties of Moreau envelopes, and empirical validation of improved performance.The paper introduces pFedMe, a personalized federated learning (FL) algorithm that addresses the challenge of statistical diversity among clients, which hinders the performance of global models. pFedMe uses Moreau envelopes as regularized loss functions to decouple personalized model optimization from global model learning. Theoretically, pFedMe achieves state-of-the-art convergence rates: quadratic speedup for strongly convex objectives and sublinear speedup of order 2/3 for smooth non-convex objectives. Experimentally, pFedMe outperforms vanilla FedAvg and Per-FedAvg in both convex and non-convex settings on real and synthetic datasets. The key contributions include the formulation of a bi-level optimization problem, leveraging the properties of Moreau envelopes, and empirical validation of improved performance.
Reach us at info@study.space