The paper introduces a heuristic method called Permutation Importance (PIMP) to correct the bias in feature importance measures, particularly in Random Forest (RF) models and Mutual Information (MI). The method normalizes the biased feature importance measures by estimating the distribution of null importances through permutations of the outcome vector. The P-value of the observed importance provides a corrected measure of feature importance. The authors demonstrate that PIMP effectively corrects the bias in RF and MI measures, improving the interpretability of the models. They apply PIMP to simulated data and real-world datasets, showing that it can successfully recover informative variables among non-informative ones and improve prediction accuracy. The improved RF model, termed PIMP-RF, is shown to outperform other models in terms of prediction accuracy. The availability of R code for PIMP is provided, and the method is argued to be applicable to any learning method that provides feature ranking.The paper introduces a heuristic method called Permutation Importance (PIMP) to correct the bias in feature importance measures, particularly in Random Forest (RF) models and Mutual Information (MI). The method normalizes the biased feature importance measures by estimating the distribution of null importances through permutations of the outcome vector. The P-value of the observed importance provides a corrected measure of feature importance. The authors demonstrate that PIMP effectively corrects the bias in RF and MI measures, improving the interpretability of the models. They apply PIMP to simulated data and real-world datasets, showing that it can successfully recover informative variables among non-informative ones and improve prediction accuracy. The improved RF model, termed PIMP-RF, is shown to outperform other models in terms of prediction accuracy. The availability of R code for PIMP is provided, and the method is argued to be applicable to any learning method that provides feature ranking.