The paper introduces FedAF, a novel aggregation-free Federated Learning (FL) algorithm designed to address data heterogeneity, particularly label-skew and feature-skew scenarios. Traditional FL methods, which aggregate local models before updating the global model, can lead to client drift and reduced model performance due to significant cross-client data heterogeneity. FedAF overcomes this issue by allowing clients to collaboratively learn condensed data using peer knowledge, which is then used by the server to train the global model. This approach enhances the quality of condensed data and improves global model performance. Extensive experiments on benchmark datasets show that FedAF outperforms various state-of-the-art FL algorithms in handling data heterogeneity, achieving superior global model accuracy and faster convergence. The paper also discusses the impact of key components of FedAF, such as collaborative data condensation and local-global knowledge matching, and provides insights into the communication cost and model size efficiency of the proposed method.The paper introduces FedAF, a novel aggregation-free Federated Learning (FL) algorithm designed to address data heterogeneity, particularly label-skew and feature-skew scenarios. Traditional FL methods, which aggregate local models before updating the global model, can lead to client drift and reduced model performance due to significant cross-client data heterogeneity. FedAF overcomes this issue by allowing clients to collaboratively learn condensed data using peer knowledge, which is then used by the server to train the global model. This approach enhances the quality of condensed data and improves global model performance. Extensive experiments on benchmark datasets show that FedAF outperforms various state-of-the-art FL algorithms in handling data heterogeneity, achieving superior global model accuracy and faster convergence. The paper also discusses the impact of key components of FedAF, such as collaborative data condensation and local-global knowledge matching, and provides insights into the communication cost and model size efficiency of the proposed method.