Recent advances in deep learning have enabled significant improvements in natural language processing (NLP) tasks, but these gains come at a high cost in terms of energy and financial resources. Training large neural networks requires substantial computational power, leading to significant carbon emissions and high costs. This paper quantifies the financial and environmental costs of training various NLP models and proposes recommendations to reduce these costs and improve equity in NLP research. The study finds that training models like BERT is equivalent to the carbon emissions of a trans-American flight. Training a single model can cost thousands of dollars in cloud computing time and electricity. Developing a new model, such as the Linguistically-Informed Self-Attention model, requires extensive computational resources, with the total GPU time equivalent to 27 years of continuous operation. These costs highlight the need for more efficient models and hardware. The paper recommends that researchers report training time and sensitivity to hyperparameters to enable direct comparisons between models. It also emphasizes the need for equitable access to computational resources, as current costs limit access for many researchers. Additionally, the paper suggests prioritizing computationally efficient algorithms and hardware, as well as developing software tools that reduce the energy required for hyperparameter tuning. The study underscores the environmental impact of NLP research and calls for a shift towards more sustainable practices. It advocates for shared computing resources and the use of efficient algorithms and hardware to reduce the carbon footprint of NLP research. The findings highlight the importance of addressing both the financial and environmental costs of NLP research to ensure equitable and sustainable development in the field.Recent advances in deep learning have enabled significant improvements in natural language processing (NLP) tasks, but these gains come at a high cost in terms of energy and financial resources. Training large neural networks requires substantial computational power, leading to significant carbon emissions and high costs. This paper quantifies the financial and environmental costs of training various NLP models and proposes recommendations to reduce these costs and improve equity in NLP research. The study finds that training models like BERT is equivalent to the carbon emissions of a trans-American flight. Training a single model can cost thousands of dollars in cloud computing time and electricity. Developing a new model, such as the Linguistically-Informed Self-Attention model, requires extensive computational resources, with the total GPU time equivalent to 27 years of continuous operation. These costs highlight the need for more efficient models and hardware. The paper recommends that researchers report training time and sensitivity to hyperparameters to enable direct comparisons between models. It also emphasizes the need for equitable access to computational resources, as current costs limit access for many researchers. Additionally, the paper suggests prioritizing computationally efficient algorithms and hardware, as well as developing software tools that reduce the energy required for hyperparameter tuning. The study underscores the environmental impact of NLP research and calls for a shift towards more sustainable practices. It advocates for shared computing resources and the use of efficient algorithms and hardware to reduce the carbon footprint of NLP research. The findings highlight the importance of addressing both the financial and environmental costs of NLP research to ensure equitable and sustainable development in the field.