3 Apr 2024 | Hongxia Li, Wei Huang, Jingya Wang, Ye Shi
The paper "Federated Prompts Cooperation via Optimal Transport for Federated Learning" addresses the challenge of integrating prompt learning into federated learning frameworks to reduce communication costs and promote local training on insufficient data. The authors introduce Federated Prompts Cooperation via Optimal Transport (FedOTP), a novel framework that learns both a global prompt for consensus among clients and a local prompt for capturing client-specific category traits. FedOTP employs unbalanced Optimal Transport (OT) to align local visual features with these prompts, balancing global consensus and local personalization. The method relaxes one of the equality constraints in classical OT, allowing prompts to focus on core image patches. Extensive experiments on datasets with various types of data heterogeneity, including label and feature shifts, demonstrate that FedOTP outperforms state-of-the-art methods. The paper also includes a theoretical analysis of the generalization bound and visualizations to illustrate the effectiveness of the proposed approach.The paper "Federated Prompts Cooperation via Optimal Transport for Federated Learning" addresses the challenge of integrating prompt learning into federated learning frameworks to reduce communication costs and promote local training on insufficient data. The authors introduce Federated Prompts Cooperation via Optimal Transport (FedOTP), a novel framework that learns both a global prompt for consensus among clients and a local prompt for capturing client-specific category traits. FedOTP employs unbalanced Optimal Transport (OT) to align local visual features with these prompts, balancing global consensus and local personalization. The method relaxes one of the equality constraints in classical OT, allowing prompts to focus on core image patches. Extensive experiments on datasets with various types of data heterogeneity, including label and feature shifts, demonstrate that FedOTP outperforms state-of-the-art methods. The paper also includes a theoretical analysis of the generalization bound and visualizations to illustrate the effectiveness of the proposed approach.