AdaFed: Fair Federated Learning via Adaptive Common Descent Direction

AdaFed: Fair Federated Learning via Adaptive Common Descent Direction

(01/2024) | Shayan Mohajer Hamidi, En-Hui Yang
Federated Learning (FL) is a promising technology where edge devices collaboratively train a machine learning model under the coordination of a server. However, learning an unfair model, where some devices are advantageously or disadvantageously treated, is a critical issue in FL. To address this problem, the authors propose AdaFed, a method that aims to find an updating direction for the server that ensures all clients' loss functions decrease, and more importantly, the loss functions with larger values decrease at a higher rate. AdaFed adaptively tunes this common direction based on local gradients and loss functions. The effectiveness of AdaFed is validated through experiments on various federated datasets, demonstrating superior performance compared to state-of-the-art fair FL methods while achieving similar prediction accuracy. The key contributions of the paper include introducing AdaFed, providing a closed-form solution for the common direction, proving convergence under different FL setups, and showing improved fairness and accuracy in experiments.Federated Learning (FL) is a promising technology where edge devices collaboratively train a machine learning model under the coordination of a server. However, learning an unfair model, where some devices are advantageously or disadvantageously treated, is a critical issue in FL. To address this problem, the authors propose AdaFed, a method that aims to find an updating direction for the server that ensures all clients' loss functions decrease, and more importantly, the loss functions with larger values decrease at a higher rate. AdaFed adaptively tunes this common direction based on local gradients and loss functions. The effectiveness of AdaFed is validated through experiments on various federated datasets, demonstrating superior performance compared to state-of-the-art fair FL methods while achieving similar prediction accuracy. The key contributions of the paper include introducing AdaFed, providing a closed-form solution for the common direction, proving convergence under different FL setups, and showing improved fairness and accuracy in experiments.
Reach us at info@study.space