Automatic Early Stopping Using Cross Validation: Quantifying the Criteria

Automatic Early Stopping Using Cross Validation: Quantifying the Criteria

December 12, 1997 | Lutz Prechelt
This paper evaluates 14 automatic stopping criteria from three classes (GL, UP, PQ) for early stopping in neural network training using cross-validation. The study aims to provide quantitative data to guide the selection of stopping criteria based on their efficiency and effectiveness in 12 different classification and approximation tasks using multi-layer perceptrons with RPROP training. The experiments show that slower stopping criteria allow for small improvements in generalization (on the order of 4%) but cost about four times longer training time. Early stopping is used to prevent overfitting by stopping training when the validation error starts to increase. However, automatic stopping criteria are often chosen ad-hoc. The study evaluates three classes of stopping criteria: GL (based on generalization loss), UP (based on changes in validation error), and PQ (based on the ratio of generalization loss and training progress). The results show that slower criteria generally lead to better generalization but require more training time. The best tradeoff between training time and generalization performance is achieved by the UP criteria, particularly UP3, UP4, and UP6. The GL criteria are preferable when multiple runs are made and the best network is selected based on validation error. The study also shows that for small networks, PQ criteria are more cost-effective for minimizing generalization error. The results indicate that the choice of stopping criterion depends on the specific problem and the desired balance between training time and generalization performance. The study provides empirical data to guide the selection of stopping criteria for neural network training.This paper evaluates 14 automatic stopping criteria from three classes (GL, UP, PQ) for early stopping in neural network training using cross-validation. The study aims to provide quantitative data to guide the selection of stopping criteria based on their efficiency and effectiveness in 12 different classification and approximation tasks using multi-layer perceptrons with RPROP training. The experiments show that slower stopping criteria allow for small improvements in generalization (on the order of 4%) but cost about four times longer training time. Early stopping is used to prevent overfitting by stopping training when the validation error starts to increase. However, automatic stopping criteria are often chosen ad-hoc. The study evaluates three classes of stopping criteria: GL (based on generalization loss), UP (based on changes in validation error), and PQ (based on the ratio of generalization loss and training progress). The results show that slower criteria generally lead to better generalization but require more training time. The best tradeoff between training time and generalization performance is achieved by the UP criteria, particularly UP3, UP4, and UP6. The GL criteria are preferable when multiple runs are made and the best network is selected based on validation error. The study also shows that for small networks, PQ criteria are more cost-effective for minimizing generalization error. The results indicate that the choice of stopping criterion depends on the specific problem and the desired balance between training time and generalization performance. The study provides empirical data to guide the selection of stopping criteria for neural network training.
Reach us at info@study.space
[slides] Automatic early stopping using cross validation%3A quantifying the criteria | StudySpace