This paper proposes a novel approach called GradSamp to reduce energy consumption in deep learning (DL) model training. The method leverages the Gaussian distribution of gradient updates in over-parameterized DL models to minimize energy usage. By sampling gradient updates from a Gaussian distribution, GradSamp enables skipping entire epochs, thereby improving training efficiency. The approach is validated across various DL models, including CNNs and transformers, for tasks such as image classification, object detection, and image segmentation. It is also applied in federated learning (FL) settings to reduce communication rounds and energy consumption. Experimental results show that GradSamp achieves significant energy savings without compromising model performance. The method is effective in both standard and out-of-distribution scenarios, including domain adaptation (DA) and domain generalization (DG). The proposed approach is simple, efficient, and applicable to a wide range of DL tasks, demonstrating its potential for practical applications in energy-efficient DL systems.This paper proposes a novel approach called GradSamp to reduce energy consumption in deep learning (DL) model training. The method leverages the Gaussian distribution of gradient updates in over-parameterized DL models to minimize energy usage. By sampling gradient updates from a Gaussian distribution, GradSamp enables skipping entire epochs, thereby improving training efficiency. The approach is validated across various DL models, including CNNs and transformers, for tasks such as image classification, object detection, and image segmentation. It is also applied in federated learning (FL) settings to reduce communication rounds and energy consumption. Experimental results show that GradSamp achieves significant energy savings without compromising model performance. The method is effective in both standard and out-of-distribution scenarios, including domain adaptation (DA) and domain generalization (DG). The proposed approach is simple, efficient, and applicable to a wide range of DL tasks, demonstrating its potential for practical applications in energy-efficient DL systems.