This paper introduces RefAug, a novel training method for language models (LMs) that integrates reflection into each math problem to enhance their reasoning abilities. Unlike traditional data expansion techniques that focus on increasing the number of training examples, RefAug targets the sequence dimension of training data by adding a reflective section to each training instance. This reflective section encourages the model to consider alternative perspectives and engage with abstractions and analogies, fostering a deeper understanding of the training problems. The method is validated through extensive experiments on diverse math reasoning tasks, demonstrating its effectiveness in improving both standard single-round question-answering settings and more complex reflective reasoning scenarios. RefAug achieves a +7.2 accuracy gain over direct fine-tuning in standard QA settings and significantly enhances performance in multiple reflective math reasoning tasks where traditional data expansion methods fall short. It also complements existing data expansion techniques, allowing for seamless integration that leads to even greater performance improvements. The method is further extended to code generation tasks, where RefAug consistently elevates the LMs' proficiency in generating accurate and reasonable code. The results show that RefAug not only enhances LMs' basic problem-solving skills but also advances their reflective reasoning abilities, making it a valuable complement to existing augmentation techniques. The paper also discusses the limitations of RefAug, including the need for higher quality data to develop models with advanced reflective math reasoning skills. Overall, RefAug demonstrates the effectiveness of incorporating reflection into training data to improve the reasoning capabilities of language models.This paper introduces RefAug, a novel training method for language models (LMs) that integrates reflection into each math problem to enhance their reasoning abilities. Unlike traditional data expansion techniques that focus on increasing the number of training examples, RefAug targets the sequence dimension of training data by adding a reflective section to each training instance. This reflective section encourages the model to consider alternative perspectives and engage with abstractions and analogies, fostering a deeper understanding of the training problems. The method is validated through extensive experiments on diverse math reasoning tasks, demonstrating its effectiveness in improving both standard single-round question-answering settings and more complex reflective reasoning scenarios. RefAug achieves a +7.2 accuracy gain over direct fine-tuning in standard QA settings and significantly enhances performance in multiple reflective math reasoning tasks where traditional data expansion methods fall short. It also complements existing data expansion techniques, allowing for seamless integration that leads to even greater performance improvements. The method is further extended to code generation tasks, where RefAug consistently elevates the LMs' proficiency in generating accurate and reasonable code. The results show that RefAug not only enhances LMs' basic problem-solving skills but also advances their reflective reasoning abilities, making it a valuable complement to existing augmentation techniques. The paper also discusses the limitations of RefAug, including the need for higher quality data to develop models with advanced reflective math reasoning skills. Overall, RefAug demonstrates the effectiveness of incorporating reflection into training data to improve the reasoning capabilities of language models.