Machine learning (ML) is increasingly used in theoretical physics and pure mathematics, where rigor and understanding are essential. This Perspective discusses techniques for achieving rigor in these fields using ML. ML techniques, while powerful, are often stochastic, error-prone, and black-box, making them challenging for rigorous scientific applications. However, they can be used to generate conjectures or verify results through reinforcement learning (RL), which can then be rigorously proven by humans. For example, ML has been used to generate conjectures in string theory, algebraic geometry, and knot theory, and to verify results in low-dimensional topology, such as the smooth 4D Poincaré conjecture. Applications of ML in theoretical physics include the use of neural networks to study field theories and metric flows. A new approach to field theory, inspired by neural network theory, and a theory of Riemannian metric flows induced by neural network gradient descent, which encompasses Perelman's formulation of the Ricci flow, are discussed. These developments leverage recent results on the statistics and dynamics of neural networks, including a correspondence between neural networks and quantum field theory (QFT), which may provide a non-perturbative definition of QFT in the continuum. ML theory has also been applied to study metric flows in Riemannian geometry, with results that include the realization of the Ricci flow as a neural network metric flow. Additionally, ML has been used in Bayesian inference and optimal transport, connecting renormalization group flows in quantum field theory to these concepts. The paper highlights the potential of ML to provide rigorous results in theoretical physics and pure mathematics, while also emphasizing the need for interpretability and the integration of ML with traditional scientific methods. It concludes with an outlook on future directions, including the application of ML theory to better understand non-perturbative quantum field theories.Machine learning (ML) is increasingly used in theoretical physics and pure mathematics, where rigor and understanding are essential. This Perspective discusses techniques for achieving rigor in these fields using ML. ML techniques, while powerful, are often stochastic, error-prone, and black-box, making them challenging for rigorous scientific applications. However, they can be used to generate conjectures or verify results through reinforcement learning (RL), which can then be rigorously proven by humans. For example, ML has been used to generate conjectures in string theory, algebraic geometry, and knot theory, and to verify results in low-dimensional topology, such as the smooth 4D Poincaré conjecture. Applications of ML in theoretical physics include the use of neural networks to study field theories and metric flows. A new approach to field theory, inspired by neural network theory, and a theory of Riemannian metric flows induced by neural network gradient descent, which encompasses Perelman's formulation of the Ricci flow, are discussed. These developments leverage recent results on the statistics and dynamics of neural networks, including a correspondence between neural networks and quantum field theory (QFT), which may provide a non-perturbative definition of QFT in the continuum. ML theory has also been applied to study metric flows in Riemannian geometry, with results that include the realization of the Ricci flow as a neural network metric flow. Additionally, ML has been used in Bayesian inference and optimal transport, connecting renormalization group flows in quantum field theory to these concepts. The paper highlights the potential of ML to provide rigorous results in theoretical physics and pure mathematics, while also emphasizing the need for interpretability and the integration of ML with traditional scientific methods. It concludes with an outlook on future directions, including the application of ML theory to better understand non-perturbative quantum field theories.