The paper reviews the application of scientific machine learning to closure problems in multiscale systems, where some quantities and processes cannot be fully prescribed despite their significant impact on simulation accuracy. It discusses different reduced model forms, distinguished by the degree of inclusion of known physics, and the objectives of a priori and a posteriori learning. The importance of adhering to physical laws in choosing the reduced model form and learning method is emphasized. The paper also reviews spatial and temporal discretization and recent trends toward discretization-invariant models. Additionally, it connects closure problems to other research disciplines such as inverse problems, Mori-Zwanzig theory, and multi-fidelity methods. While progress has been made, challenges remain, particularly in the generalizability and interpretability of learned models. The paper concludes by highlighting future research directions.The paper reviews the application of scientific machine learning to closure problems in multiscale systems, where some quantities and processes cannot be fully prescribed despite their significant impact on simulation accuracy. It discusses different reduced model forms, distinguished by the degree of inclusion of known physics, and the objectives of a priori and a posteriori learning. The importance of adhering to physical laws in choosing the reduced model form and learning method is emphasized. The paper also reviews spatial and temporal discretization and recent trends toward discretization-invariant models. Additionally, it connects closure problems to other research disciplines such as inverse problems, Mori-Zwanzig theory, and multi-fidelity methods. While progress has been made, challenges remain, particularly in the generalizability and interpretability of learned models. The paper concludes by highlighting future research directions.