The paper "Towards A Rigorous Science of Interpretable Machine Learning" by Finale Doshi-Velez and Been Kim discusses the importance of interpretability in machine learning systems. As ML systems become more prevalent in areas like autonomous vehicles, email filters, and predictive policing, there is a growing need for these systems to be not only accurate but also safe, fair, and explainable. Interpretability is crucial for ensuring that ML systems operate safely and ethically, especially in complex applications where the consequences of errors can be severe. The paper highlights the lack of consensus on what interpretability means and how to evaluate it. Current approaches to evaluating interpretability fall into two categories: application-based evaluations, which assess whether a system is useful in a practical context, and proxy-based evaluations, which use quantifiable measures like model sparsity to assess interpretability. However, these approaches often rely on subjective notions of "you'll know it when you see it," which can lead to inconsistent evaluations. The authors propose a taxonomy of interpretability evaluation: application-grounded, human-grounded, and functionally-grounded. Application-grounded evaluation involves testing models in real-world scenarios, while human-grounded evaluation uses simplified tasks to assess the quality of explanations. Functionally-grounded evaluation uses formal definitions of interpretability as a proxy for explanation quality. The paper also discusses open problems in the science of interpretability, including determining the best proxies for different applications, identifying important factors in designing simplified tasks, and characterizing proxies for explanation quality. The authors suggest a data-driven approach to discover factors of interpretability by creating a matrix of real-world tasks and methods, which can help predict which methods are most promising for new problems. In conclusion, the paper calls for a rigorous and evidence-based approach to evaluating interpretability in machine learning systems. This includes developing a shared language and taxonomy for interpretability, as well as considering the specific needs of different applications and users. The authors emphasize the importance of human-subject experiments and the need for careful experimental design to ensure that evaluations are meaningful and reliable.The paper "Towards A Rigorous Science of Interpretable Machine Learning" by Finale Doshi-Velez and Been Kim discusses the importance of interpretability in machine learning systems. As ML systems become more prevalent in areas like autonomous vehicles, email filters, and predictive policing, there is a growing need for these systems to be not only accurate but also safe, fair, and explainable. Interpretability is crucial for ensuring that ML systems operate safely and ethically, especially in complex applications where the consequences of errors can be severe. The paper highlights the lack of consensus on what interpretability means and how to evaluate it. Current approaches to evaluating interpretability fall into two categories: application-based evaluations, which assess whether a system is useful in a practical context, and proxy-based evaluations, which use quantifiable measures like model sparsity to assess interpretability. However, these approaches often rely on subjective notions of "you'll know it when you see it," which can lead to inconsistent evaluations. The authors propose a taxonomy of interpretability evaluation: application-grounded, human-grounded, and functionally-grounded. Application-grounded evaluation involves testing models in real-world scenarios, while human-grounded evaluation uses simplified tasks to assess the quality of explanations. Functionally-grounded evaluation uses formal definitions of interpretability as a proxy for explanation quality. The paper also discusses open problems in the science of interpretability, including determining the best proxies for different applications, identifying important factors in designing simplified tasks, and characterizing proxies for explanation quality. The authors suggest a data-driven approach to discover factors of interpretability by creating a matrix of real-world tasks and methods, which can help predict which methods are most promising for new problems. In conclusion, the paper calls for a rigorous and evidence-based approach to evaluating interpretability in machine learning systems. This includes developing a shared language and taxonomy for interpretability, as well as considering the specific needs of different applications and users. The authors emphasize the importance of human-subject experiments and the need for careful experimental design to ensure that evaluations are meaningful and reliable.