This paper introduces FederatedAveraging (FedAvg), a practical method for federated learning of deep networks that is robust to unbalanced and non-IID data distributions. Federated learning allows training on distributed data without centralizing it, which is crucial for privacy-sensitive data. FedAvg combines local SGD training on each client with communication rounds where the central server performs model averaging. The key insight is that parameter averaging over updates from multiple clients produces surprisingly good results, significantly reducing communication needed to train deep networks. The paper discusses the advantages of federated learning, including privacy benefits and reduced communication costs for large datasets. It also addresses the challenges of federated optimization, such as non-IID data distributions and unbalanced client data. The paper presents experiments showing that FedAvg can train high-quality models in relatively few communication rounds, with significant speedups compared to traditional methods. It also explores the effectiveness of FedAvg on image classification and language modeling tasks, demonstrating its robustness to non-IID data and its ability to achieve high test accuracy. The paper concludes that federated learning has significant promise for training high-quality models with minimal communication, and suggests future work in exploring compatibility with other optimization algorithms and model structures.This paper introduces FederatedAveraging (FedAvg), a practical method for federated learning of deep networks that is robust to unbalanced and non-IID data distributions. Federated learning allows training on distributed data without centralizing it, which is crucial for privacy-sensitive data. FedAvg combines local SGD training on each client with communication rounds where the central server performs model averaging. The key insight is that parameter averaging over updates from multiple clients produces surprisingly good results, significantly reducing communication needed to train deep networks. The paper discusses the advantages of federated learning, including privacy benefits and reduced communication costs for large datasets. It also addresses the challenges of federated optimization, such as non-IID data distributions and unbalanced client data. The paper presents experiments showing that FedAvg can train high-quality models in relatively few communication rounds, with significant speedups compared to traditional methods. It also explores the effectiveness of FedAvg on image classification and language modeling tasks, demonstrating its robustness to non-IID data and its ability to achieve high test accuracy. The paper concludes that federated learning has significant promise for training high-quality models with minimal communication, and suggests future work in exploring compatibility with other optimization algorithms and model structures.