14 Mar 2024 | Zhenheng Tang, Yonggang Zhang, Shaohuai Shi, Xinmei Tian, Tongliang Liu, Bo Han, Xiaowen Chu
The paper "FedImpro: Measuring and Improving Client Update in Federated Learning" addresses the issue of client drift in Federated Learning (FL) caused by heterogeneous data distributions across clients. The authors propose a novel approach, FedImpro, which aims to mitigate client drift by generating improved local models. They analyze the generalization contribution of local training and conclude that it is bounded by the conditional Wasserstein distance between client data distributions. FedImpro decouples the model into high-level and low-level components, training the high-level portion on reconstructed feature distributions to enhance generalization and reduce gradient dissimilarity. Experimental results demonstrate that FedImpro effectively improves the generalization performance of FL models and reduces gradient dissimilarity, even under severe data heterogeneity. The paper also includes a theoretical analysis and extensive experiments to validate the effectiveness of FedImpro.The paper "FedImpro: Measuring and Improving Client Update in Federated Learning" addresses the issue of client drift in Federated Learning (FL) caused by heterogeneous data distributions across clients. The authors propose a novel approach, FedImpro, which aims to mitigate client drift by generating improved local models. They analyze the generalization contribution of local training and conclude that it is bounded by the conditional Wasserstein distance between client data distributions. FedImpro decouples the model into high-level and low-level components, training the high-level portion on reconstructed feature distributions to enhance generalization and reduce gradient dissimilarity. Experimental results demonstrate that FedImpro effectively improves the generalization performance of FL models and reduces gradient dissimilarity, even under severe data heterogeneity. The paper also includes a theoretical analysis and extensive experiments to validate the effectiveness of FedImpro.