The paper introduces InternLM-Math, an open-source large language model (LLM) designed for mathematical reasoning. InternLM-Math is based on the InternLM2-Base model and is pre-trained on a diverse collection of high-quality data, including math corpora, domain-specific datasets, and synthetic data. The model is trained to perform various mathematical tasks, such as solving problems, verifying solutions, proving statements, and using code interpreters. It achieves state-of-the-art performance on multiple benchmarks, including GSM8K, MATH, Hungary math exam, MathBench-ZH, and MiniF2F. The paper also explores the use of the formal math language LEAN for solving and proving math problems, demonstrating the potential of using LEAN as a unified platform for math reasoning. The authors provide detailed descriptions of the pre-training data composition, training strategy, and evaluation results, highlighting the model's strengths and limitations. The paper concludes by discussing future directions, including improving chain-of-thought reasoning, adding self-critique capabilities, and enhancing process reward modeling.The paper introduces InternLM-Math, an open-source large language model (LLM) designed for mathematical reasoning. InternLM-Math is based on the InternLM2-Base model and is pre-trained on a diverse collection of high-quality data, including math corpora, domain-specific datasets, and synthetic data. The model is trained to perform various mathematical tasks, such as solving problems, verifying solutions, proving statements, and using code interpreters. It achieves state-of-the-art performance on multiple benchmarks, including GSM8K, MATH, Hungary math exam, MathBench-ZH, and MiniF2F. The paper also explores the use of the formal math language LEAN for solving and proving math problems, demonstrating the potential of using LEAN as a unified platform for math reasoning. The authors provide detailed descriptions of the pre-training data composition, training strategy, and evaluation results, highlighting the model's strengths and limitations. The paper concludes by discussing future directions, including improving chain-of-thought reasoning, adding self-critique capabilities, and enhancing process reward modeling.