7 Mar 2019 | Scott M. Lundberg, Gabriel G. Erion, and Su-In Lee
This paper introduces SHAP (SHapley Additive explanation) values as a consistent and locally accurate method for feature attribution in tree ensembles. The authors show that existing feature attribution methods for tree ensembles are inconsistent, meaning they can lower a feature's assigned importance when the true impact of that feature increases. SHAP values are based on game theory and provide a unique consistent and locally accurate attribution method. They are also extended to interaction effects, defining SHAP interaction values. The authors propose a rich visualization of individualized feature attributions that improves over classic attribution summaries and partial dependence plots, and a unique "supervised" clustering method. They demonstrate better agreement with human intuition through a user study, exponential improvements in run time, improved clustering performance, and better identification of influential features. An implementation of their algorithm has been merged into XGBoost and LightGBM. The paper also presents Tree SHAP, a fast algorithm for computing SHAP values for tree ensembles, and SHAP interaction values for pairwise interaction effects. The authors show that SHAP values provide a strict theoretical improvement by eliminating significant consistency problems. They also demonstrate the effectiveness of SHAP values through experiments and applications, including supervised clustering, SHAP summary plots, and SHAP dependence plots. The paper concludes that SHAP values are a practical replacement for previous methods and provide a better understanding of tree models.This paper introduces SHAP (SHapley Additive explanation) values as a consistent and locally accurate method for feature attribution in tree ensembles. The authors show that existing feature attribution methods for tree ensembles are inconsistent, meaning they can lower a feature's assigned importance when the true impact of that feature increases. SHAP values are based on game theory and provide a unique consistent and locally accurate attribution method. They are also extended to interaction effects, defining SHAP interaction values. The authors propose a rich visualization of individualized feature attributions that improves over classic attribution summaries and partial dependence plots, and a unique "supervised" clustering method. They demonstrate better agreement with human intuition through a user study, exponential improvements in run time, improved clustering performance, and better identification of influential features. An implementation of their algorithm has been merged into XGBoost and LightGBM. The paper also presents Tree SHAP, a fast algorithm for computing SHAP values for tree ensembles, and SHAP interaction values for pairwise interaction effects. The authors show that SHAP values provide a strict theoretical improvement by eliminating significant consistency problems. They also demonstrate the effectiveness of SHAP values through experiments and applications, including supervised clustering, SHAP summary plots, and SHAP dependence plots. The paper concludes that SHAP values are a practical replacement for previous methods and provide a better understanding of tree models.