RGA: a unified measure of predictive accuracy

RGA: a unified measure of predictive accuracy

17 January 2024 | Paolo Giudici, Emanuela Raffinetti
The paper introduces the Rank Graduation Accuracy (RGA) measure, a unified framework for evaluating the predictive accuracy of statistical forecasts. RGA is designed to be applicable to various types of response variables, including continuous, ordinal, and binary, by focusing on the concordance between the ranks of predicted and actual values. The authors demonstrate that RGA coincides with the Area Under the ROC Curve (AUROC) and the Wilcoxon-Mann–Whitney statistic in the binary case, and can be used to evaluate the accuracy of probability forecasts. In the real-valued case, RGA can be applied similarly to measures like Root Mean Square Error (RMSE), but it evaluates point predictions based on their ranks rather than their values, enhancing robustness. The paper also discusses the statistical properties of RGA, including normalization, invariance under positive affine transformations, and equivalence with AUROC and the Wilcoxon-Mann–Whitney statistic. The effectiveness of RGA is demonstrated through simulations and real data analysis, showing its ability to select the best models and predict the correct ordering of responses. The authors conclude by highlighting the importance of RGA in evaluating the accuracy of machine learning models, especially in the context of Artificial Intelligence (AI) applications, where it can help assess key principles such as sustainability, accuracy, fairness, and explainability.The paper introduces the Rank Graduation Accuracy (RGA) measure, a unified framework for evaluating the predictive accuracy of statistical forecasts. RGA is designed to be applicable to various types of response variables, including continuous, ordinal, and binary, by focusing on the concordance between the ranks of predicted and actual values. The authors demonstrate that RGA coincides with the Area Under the ROC Curve (AUROC) and the Wilcoxon-Mann–Whitney statistic in the binary case, and can be used to evaluate the accuracy of probability forecasts. In the real-valued case, RGA can be applied similarly to measures like Root Mean Square Error (RMSE), but it evaluates point predictions based on their ranks rather than their values, enhancing robustness. The paper also discusses the statistical properties of RGA, including normalization, invariance under positive affine transformations, and equivalence with AUROC and the Wilcoxon-Mann–Whitney statistic. The effectiveness of RGA is demonstrated through simulations and real data analysis, showing its ability to select the best models and predict the correct ordering of responses. The authors conclude by highlighting the importance of RGA in evaluating the accuracy of machine learning models, especially in the context of Artificial Intelligence (AI) applications, where it can help assess key principles such as sustainability, accuracy, fairness, and explainability.
Reach us at info@study.space
[slides] RGA%3A a unified measure of predictive accuracy | StudySpace