This paper reviews scientific machine learning approaches for solving closure problems in multiscale systems. Closure problems arise when certain quantities and processes cannot be fully prescribed despite their effects on simulation accuracy. Scientific machine learning combines traditional physics-based modeling with data-driven techniques, often through enriching differential equations with neural networks. The paper discusses different reduced model forms, distinguishing by the degree of known physics inclusion and the objectives of a priori and a posteriori learning. It emphasizes the importance of adhering to physical laws in choosing reduced models and learning methods. The effects of spatial and temporal discretization and recent trends toward discretization-invariant models are reviewed. Connections are made between closure problems and other disciplines like inverse problems, Mori-Zwanzig theory, and multi-fidelity methods. The paper highlights progress in scientific machine learning for closure problems but notes remaining challenges, particularly in generalizability and interpretability of learned models. It discusses various reduced model forms, including closure models, evolution forms, and surrogate models. The paper also covers a priori and a posteriori learning approaches, emphasizing the importance of minimizing residual or solution errors. It discusses the use of neural networks for closure models, the challenges of stability, and the potential of reinforcement learning. The paper concludes that while scientific machine learning has made significant progress, challenges remain in ensuring model accuracy, stability, and interpretability.This paper reviews scientific machine learning approaches for solving closure problems in multiscale systems. Closure problems arise when certain quantities and processes cannot be fully prescribed despite their effects on simulation accuracy. Scientific machine learning combines traditional physics-based modeling with data-driven techniques, often through enriching differential equations with neural networks. The paper discusses different reduced model forms, distinguishing by the degree of known physics inclusion and the objectives of a priori and a posteriori learning. It emphasizes the importance of adhering to physical laws in choosing reduced models and learning methods. The effects of spatial and temporal discretization and recent trends toward discretization-invariant models are reviewed. Connections are made between closure problems and other disciplines like inverse problems, Mori-Zwanzig theory, and multi-fidelity methods. The paper highlights progress in scientific machine learning for closure problems but notes remaining challenges, particularly in generalizability and interpretability of learned models. It discusses various reduced model forms, including closure models, evolution forms, and surrogate models. The paper also covers a priori and a posteriori learning approaches, emphasizing the importance of minimizing residual or solution errors. It discusses the use of neural networks for closure models, the challenges of stability, and the potential of reinforcement learning. The paper concludes that while scientific machine learning has made significant progress, challenges remain in ensuring model accuracy, stability, and interpretability.