Global and Local Prompts Cooperation via Optimal Transport for Federated Learning

Global and Local Prompts Cooperation via Optimal Transport for Federated Learning

3 Apr 2024 | Hongxia Li¹ Wei Huang² Jingya Wang¹ Ye Shi¹,*
This paper proposes FedOTP, a federated learning framework that integrates optimal transport (OT) to enhance the cooperation between global and local prompts, addressing data heterogeneity in federated learning. The framework introduces a global prompt to capture consensus knowledge across clients and a local prompt to capture client-specific category characteristics. Unbalanced OT is used to align local visual features with these prompts, balancing global consensus and local personalization. By relaxing one of the equality constraints in OT, FedOTP enables prompts to focus solely on the most relevant image patches, improving performance. Extensive experiments on datasets with various types of heterogeneities demonstrate that FedOTP outperforms state-of-the-art methods. The framework effectively handles both label shift and feature shift data heterogeneity, achieving superior performance in diverse scenarios. Visualization results confirm that global prompts focus on common features, while local prompts capture client-specific details. The method is robust across different data distributions and client numbers, showing strong generalization capabilities. The paper also provides theoretical analysis of the generalization bound of FedOTP, demonstrating its effectiveness in handling data heterogeneity.This paper proposes FedOTP, a federated learning framework that integrates optimal transport (OT) to enhance the cooperation between global and local prompts, addressing data heterogeneity in federated learning. The framework introduces a global prompt to capture consensus knowledge across clients and a local prompt to capture client-specific category characteristics. Unbalanced OT is used to align local visual features with these prompts, balancing global consensus and local personalization. By relaxing one of the equality constraints in OT, FedOTP enables prompts to focus solely on the most relevant image patches, improving performance. Extensive experiments on datasets with various types of heterogeneities demonstrate that FedOTP outperforms state-of-the-art methods. The framework effectively handles both label shift and feature shift data heterogeneity, achieving superior performance in diverse scenarios. Visualization results confirm that global prompts focus on common features, while local prompts capture client-specific details. The method is robust across different data distributions and client numbers, showing strong generalization capabilities. The paper also provides theoretical analysis of the generalization bound of FedOTP, demonstrating its effectiveness in handling data heterogeneity.
Reach us at info@study.space