FedAC: An Adaptive Clustered Federated Learning Framework for Heterogeneous Data

FedAC: An Adaptive Clustered Federated Learning Framework for Heterogeneous Data

29 Mar 2024 | Yuxin Zhang, Haoyu Chen, Zheng Lin, Zhe Chen, and Jin Zhao
FedAC is an adaptive clustered federated learning framework designed to address data heterogeneity in federated learning (FL). The framework decouples neural networks into submodules, enabling efficient integration of global knowledge into intra-cluster learning through distinct aggregation methods. It also introduces a cost-effective online model similarity metric based on dimensionality reduction and a cluster number fine-tuning module for improved adaptability and scalability in complex, heterogeneous environments. Extensive experiments show that FedAC outperforms state-of-the-art methods, achieving a 1.82% and 12.67% increase in test accuracy on CIFAR-10 and CIFAR-100 datasets, respectively, under different non-IID settings. FedAC addresses the challenges of data heterogeneity in FL by clustering clients into groups with higher internal similarity. It employs a low-rank cosine similarity metric for efficient model similarity measurement and an EM-like algorithm for periodic clustering updates. The framework dynamically adjusts the cluster count based on the current clustering status, eliminating the need for a fixed hyperparameter. This adaptability enhances the system's robustness and flexibility. The framework's effectiveness is validated through ablation experiments, demonstrating its flexibility and robustness. FedAC's performance is evaluated on diverse datasets in complex heterogeneous scenarios, showing superior overall performance compared to SOTA methods. The framework's components, including the low-rank cosine similarity metric and cluster number tuning module, are shown to be effective in improving model performance and clustering effectiveness. The results highlight FedAC's ability to handle complex, heterogeneous environments and achieve optimal performance through adaptive clustering and model regularization.FedAC is an adaptive clustered federated learning framework designed to address data heterogeneity in federated learning (FL). The framework decouples neural networks into submodules, enabling efficient integration of global knowledge into intra-cluster learning through distinct aggregation methods. It also introduces a cost-effective online model similarity metric based on dimensionality reduction and a cluster number fine-tuning module for improved adaptability and scalability in complex, heterogeneous environments. Extensive experiments show that FedAC outperforms state-of-the-art methods, achieving a 1.82% and 12.67% increase in test accuracy on CIFAR-10 and CIFAR-100 datasets, respectively, under different non-IID settings. FedAC addresses the challenges of data heterogeneity in FL by clustering clients into groups with higher internal similarity. It employs a low-rank cosine similarity metric for efficient model similarity measurement and an EM-like algorithm for periodic clustering updates. The framework dynamically adjusts the cluster count based on the current clustering status, eliminating the need for a fixed hyperparameter. This adaptability enhances the system's robustness and flexibility. The framework's effectiveness is validated through ablation experiments, demonstrating its flexibility and robustness. FedAC's performance is evaluated on diverse datasets in complex heterogeneous scenarios, showing superior overall performance compared to SOTA methods. The framework's components, including the low-rank cosine similarity metric and cluster number tuning module, are shown to be effective in improving model performance and clustering effectiveness. The results highlight FedAC's ability to handle complex, heterogeneous environments and achieve optimal performance through adaptive clustering and model regularization.
Reach us at info@study.space