The paper introduces a novel optimization approach called Schedule-Free learning, which eliminates the need for specifying a learning rate schedule while achieving or surpassing the performance of schedule-based methods. The approach is designed to address the gap between theoretical guarantees and practical performance in optimization, particularly in the context of stochastic gradient descent (SGD) and large-scale deep learning problems. Key contributions include: 1. **Avoiding Stopping Time Specification**: Unlike traditional learning rate schedules, Schedule-Free learning does not require the stopping time \( T \) to be known in advance, making it more flexible and practical. 2. **State-of-the-Art Performance**: The method matches or outperforms existing schedules across a wide range of problems, including convex optimization and large-scale deep learning tasks. 3. **No Additional Hyperparameters**: The approach introduces no additional hyperparameters beyond those required for standard optimizers with momentum. 4. **Theoretical Foundations**: The paper develops a new theory that unifies scheduling and iterate averaging, providing theoretical guarantees for the method's performance. 5. **Comprehensive Evaluation**: Extensive experiments across various domains (computer vision, language, and categorical data) demonstrate the effectiveness of Schedule-Free learning, showing strong performance compared to state-of-the-art schedules. The method is implemented in an open-source framework and is evaluated on a large-scale benchmark, covering a wide range of problems from small-scale to large-scale deep learning tasks. The results highlight the method's ability to achieve faster convergence and better performance with larger learning rates, even in non-convex optimization problems.The paper introduces a novel optimization approach called Schedule-Free learning, which eliminates the need for specifying a learning rate schedule while achieving or surpassing the performance of schedule-based methods. The approach is designed to address the gap between theoretical guarantees and practical performance in optimization, particularly in the context of stochastic gradient descent (SGD) and large-scale deep learning problems. Key contributions include: 1. **Avoiding Stopping Time Specification**: Unlike traditional learning rate schedules, Schedule-Free learning does not require the stopping time \( T \) to be known in advance, making it more flexible and practical. 2. **State-of-the-Art Performance**: The method matches or outperforms existing schedules across a wide range of problems, including convex optimization and large-scale deep learning tasks. 3. **No Additional Hyperparameters**: The approach introduces no additional hyperparameters beyond those required for standard optimizers with momentum. 4. **Theoretical Foundations**: The paper develops a new theory that unifies scheduling and iterate averaging, providing theoretical guarantees for the method's performance. 5. **Comprehensive Evaluation**: Extensive experiments across various domains (computer vision, language, and categorical data) demonstrate the effectiveness of Schedule-Free learning, showing strong performance compared to state-of-the-art schedules. The method is implemented in an open-source framework and is evaluated on a large-scale benchmark, covering a wide range of problems from small-scale to large-scale deep learning tasks. The results highlight the method's ability to achieve faster convergence and better performance with larger learning rates, even in non-convex optimization problems.