LEAF is a modular benchmarking framework for learning in federated settings. It includes a suite of open-source federated datasets, a rigorous evaluation framework, and a set of reference implementations, all aimed at capturing the challenges of practical federated environments. The framework addresses the statistical, systems, and privacy and security challenges inherent in federated learning, including data heterogeneity, communication bottlenecks, and privacy concerns. LEAF provides realistic federated datasets that reflect the diversity and scale of real-world federated scenarios, as well as metrics for evaluating performance across devices and systems. It also includes reference implementations of algorithms for federated learning, such as minibatch SGD, FedAvg, and Mocha. LEAF enables reproducible science by providing standardized datasets and metrics, and it supports a wide range of learning paradigms, including federated learning, meta-learning, multi-task learning, and on-device learning. The framework is designed to be modular, allowing for easy integration into diverse experimental pipelines. LEAF aims to bridge the gap between popular, artificial datasets and those that realistically capture federated scenarios, while also providing a clear methodology for evaluating and reproducing results. The framework includes datasets such as Federated Extended MNIST, Sentiment140, Shakespeare, CelebA, Reddit, and a synthetic dataset. It also provides statistical and systems metrics for evaluating performance, including performance at different percentiles, computational resource usage, and accuracy weighting. LEAF is intended to foster informed and grounded progress in federated learning and related areas by providing a realistic benchmarking framework.LEAF is a modular benchmarking framework for learning in federated settings. It includes a suite of open-source federated datasets, a rigorous evaluation framework, and a set of reference implementations, all aimed at capturing the challenges of practical federated environments. The framework addresses the statistical, systems, and privacy and security challenges inherent in federated learning, including data heterogeneity, communication bottlenecks, and privacy concerns. LEAF provides realistic federated datasets that reflect the diversity and scale of real-world federated scenarios, as well as metrics for evaluating performance across devices and systems. It also includes reference implementations of algorithms for federated learning, such as minibatch SGD, FedAvg, and Mocha. LEAF enables reproducible science by providing standardized datasets and metrics, and it supports a wide range of learning paradigms, including federated learning, meta-learning, multi-task learning, and on-device learning. The framework is designed to be modular, allowing for easy integration into diverse experimental pipelines. LEAF aims to bridge the gap between popular, artificial datasets and those that realistically capture federated scenarios, while also providing a clear methodology for evaluating and reproducing results. The framework includes datasets such as Federated Extended MNIST, Sentiment140, Shakespeare, CelebA, Reddit, and a synthetic dataset. It also provides statistical and systems metrics for evaluating performance, including performance at different percentiles, computational resource usage, and accuracy weighting. LEAF is intended to foster informed and grounded progress in federated learning and related areas by providing a realistic benchmarking framework.