Ditto: Fair and Robust Federated Learning Through Personalization

Ditto: Fair and Robust Federated Learning Through Personalization

15 Jun 2021 | Tian Li, Shengyuan Hu, Ahmad Beirami, Virginia Smith
The paper "Ditto: Fair and Robust Federated Learning Through Personalization" addresses the challenges of fairness and robustness in federated learning systems. It identifies that these two goals are often competing constraints in statistically heterogeneous networks. To tackle this, the authors propose Ditto, a scalable framework for personalized federated learning that inherently provides both fairness and robustness benefits. The framework is designed to be a lightweight add-on for standard global federated learning, applicable to both convex and non-convex objectives, while retaining the privacy and efficiency properties of traditional FL. The authors theoretically analyze Ditto's ability to achieve fairness and robustness simultaneously on a class of linear problems. Empirically, Ditto is evaluated across a suite of federated datasets, demonstrating competitive performance relative to recent personalization methods and superior results compared to state-of-the-art fair or robust baselines. The key contributions of the paper include: 1. **Proposition of Ditto**: A multi-task learning objective for federated learning that incorporates personalization while maintaining efficiency and privacy. 2. **Theoretical Analysis**: Convergence guarantees for the Ditto solver, which incorporate common practices in cross-device federated learning. 3. **Empirical Results**: Superior accuracy, robustness, and fairness compared to state-of-the-art methods, both in terms of accuracy and robustness against training-time attacks. 4. **Practical Benefits**: Modularity and practical advantages in terms of privacy and efficiency. The paper also discusses related work, including robustness and fairness in federated learning, and personalization techniques. It highlights the tension between fairness and robustness and how Ditto addresses this by leveraging personalized models to improve overall performance.The paper "Ditto: Fair and Robust Federated Learning Through Personalization" addresses the challenges of fairness and robustness in federated learning systems. It identifies that these two goals are often competing constraints in statistically heterogeneous networks. To tackle this, the authors propose Ditto, a scalable framework for personalized federated learning that inherently provides both fairness and robustness benefits. The framework is designed to be a lightweight add-on for standard global federated learning, applicable to both convex and non-convex objectives, while retaining the privacy and efficiency properties of traditional FL. The authors theoretically analyze Ditto's ability to achieve fairness and robustness simultaneously on a class of linear problems. Empirically, Ditto is evaluated across a suite of federated datasets, demonstrating competitive performance relative to recent personalization methods and superior results compared to state-of-the-art fair or robust baselines. The key contributions of the paper include: 1. **Proposition of Ditto**: A multi-task learning objective for federated learning that incorporates personalization while maintaining efficiency and privacy. 2. **Theoretical Analysis**: Convergence guarantees for the Ditto solver, which incorporate common practices in cross-device federated learning. 3. **Empirical Results**: Superior accuracy, robustness, and fairness compared to state-of-the-art methods, both in terms of accuracy and robustness against training-time attacks. 4. **Practical Benefits**: Modularity and practical advantages in terms of privacy and efficiency. The paper also discusses related work, including robustness and fairness in federated learning, and personalization techniques. It highlights the tension between fairness and robustness and how Ditto addresses this by leveraging personalized models to improve overall performance.
Reach us at info@study.space
Understanding Ditto%3A Fair and Robust Federated Learning Through Personalization