A Survey of Methods for Explaining Black Box Models

A Survey of Methods for Explaining Black Box Models

August 2018 | RICCARDO GUIDOTTI, ANNA MONREALE, SALVATORE RUGGIERI, FRANCO TURINI, FOSCA GIANNOTTI, DINO PEDRESCHI
A Survey of Methods for Explaining Black Box Models This paper surveys methods for explaining black box models, which are decision systems that hide their internal logic. These systems are increasingly used in decision support, but lack of explanation poses practical and ethical issues. The paper classifies the main problems addressed in the literature regarding explanation and black box systems, aiming to help researchers find useful proposals for their work. It also discusses open research questions and future directions. Black box systems, such as machine learning models, can inherit biases from training data, leading to unfair decisions. The GDPR introduced a right to explanation for individuals, emphasizing the need for interpretable models. However, current methods often sacrifice accuracy for interpretability, and there is a lack of systematic classification of these methods. The paper discusses the need for interpretable models, highlighting real-world cases where black boxes can be dangerous. It defines interpretability as the ability to explain or provide meaning in understandable terms to humans. It also discusses the dimensions of interpretability, including global and local interpretability, time limitations, and user expertise. The paper presents three main problems: model explanation, outcome explanation, and model inspection. Model explanation involves providing a global explanation of the black box model. Outcome explanation focuses on explaining the decision for a specific input instance. Model inspection involves understanding specific properties of the black box model or its predictions. The paper also discusses the complexity of interpretable models, the types of data used for classification, and the challenges of explaining black box models for different data types. It concludes that a clear classification of these methods is needed to organize the body of knowledge and address open research questions.A Survey of Methods for Explaining Black Box Models This paper surveys methods for explaining black box models, which are decision systems that hide their internal logic. These systems are increasingly used in decision support, but lack of explanation poses practical and ethical issues. The paper classifies the main problems addressed in the literature regarding explanation and black box systems, aiming to help researchers find useful proposals for their work. It also discusses open research questions and future directions. Black box systems, such as machine learning models, can inherit biases from training data, leading to unfair decisions. The GDPR introduced a right to explanation for individuals, emphasizing the need for interpretable models. However, current methods often sacrifice accuracy for interpretability, and there is a lack of systematic classification of these methods. The paper discusses the need for interpretable models, highlighting real-world cases where black boxes can be dangerous. It defines interpretability as the ability to explain or provide meaning in understandable terms to humans. It also discusses the dimensions of interpretability, including global and local interpretability, time limitations, and user expertise. The paper presents three main problems: model explanation, outcome explanation, and model inspection. Model explanation involves providing a global explanation of the black box model. Outcome explanation focuses on explaining the decision for a specific input instance. Model inspection involves understanding specific properties of the black box model or its predictions. The paper also discusses the complexity of interpretable models, the types of data used for classification, and the challenges of explaining black box models for different data types. It concludes that a clear classification of these methods is needed to organize the body of knowledge and address open research questions.
Reach us at info@study.space