The paper introduces LEAF, a modular benchmarking framework designed for learning in federated settings. Federated networks, such as those comprising wearable devices, mobile phones, or autonomous vehicles, generate vast amounts of data daily, which can be leveraged to improve user experiences. However, the scale and heterogeneity of federated data pose significant challenges in areas like federated learning, meta-learning, and multi-task learning. To address these challenges, LEAF provides a suite of open-source federated datasets, a rigorous evaluation framework, and reference implementations. The framework aims to bridge the gap between artificial datasets commonly used for benchmarking and realistic federated scenarios. LEAF includes six datasets, introduces metrics that capture performance across devices and system resource usage, and offers reference implementations of federated learning algorithms. The paper demonstrates LEAF's capabilities through reproducible experiments, granular metrics, and modular design, showcasing its potential to foster informed and grounded progress in federated learning research.The paper introduces LEAF, a modular benchmarking framework designed for learning in federated settings. Federated networks, such as those comprising wearable devices, mobile phones, or autonomous vehicles, generate vast amounts of data daily, which can be leveraged to improve user experiences. However, the scale and heterogeneity of federated data pose significant challenges in areas like federated learning, meta-learning, and multi-task learning. To address these challenges, LEAF provides a suite of open-source federated datasets, a rigorous evaluation framework, and reference implementations. The framework aims to bridge the gap between artificial datasets commonly used for benchmarking and realistic federated scenarios. LEAF includes six datasets, introduces metrics that capture performance across devices and system resource usage, and offers reference implementations of federated learning algorithms. The paper demonstrates LEAF's capabilities through reproducible experiments, granular metrics, and modular design, showcasing its potential to foster informed and grounded progress in federated learning research.