This paper introduces a method to explain the decisions of any classification algorithm by estimating local explanation vectors. These vectors provide insights into the features that influence the prediction of individual data points. The method is based on the concept of local gradients of the probability function, which can be applied to various classifiers, including Gaussian Process Classification (GPC), k-Nearest Neighbors (k-NN), and Support Vector Machines (SVM). The paper first defines local explanation vectors as gradients of the conditional probability of the true label being different from the predicted label. These vectors are then applied to different classification tasks, such as Iris flower classification and USPS digit classification. For GPC, the local gradients are derived analytically, while for other classifiers, they are estimated using Parzen window methods. The results show that the explanation vectors can highlight the features that are most influential for the prediction of each data point. The method is also applied to a real-world drug discovery problem, where the explanation vectors help identify the features that make certain compounds mutagenic or non-mutagenic. The results align with established knowledge about toxicophores and detoxicophores, demonstrating the effectiveness of the approach in capturing domain-specific insights. The paper discusses the limitations of the method, including cases where the derivative is zero, and the behavior of explanation vectors near the boundaries of the training data. It also addresses the assumption of stationarity in the data and the need for appropriate methods when dealing with non-stationary data. In conclusion, the proposed method provides a way to understand the decisions made by complex classifiers, enabling a deeper understanding of the underlying patterns and features that influence predictions. This approach is particularly useful in applications where interpretability is crucial, such as in drug discovery and other domains where the reasoning behind predictions needs to be transparent.This paper introduces a method to explain the decisions of any classification algorithm by estimating local explanation vectors. These vectors provide insights into the features that influence the prediction of individual data points. The method is based on the concept of local gradients of the probability function, which can be applied to various classifiers, including Gaussian Process Classification (GPC), k-Nearest Neighbors (k-NN), and Support Vector Machines (SVM). The paper first defines local explanation vectors as gradients of the conditional probability of the true label being different from the predicted label. These vectors are then applied to different classification tasks, such as Iris flower classification and USPS digit classification. For GPC, the local gradients are derived analytically, while for other classifiers, they are estimated using Parzen window methods. The results show that the explanation vectors can highlight the features that are most influential for the prediction of each data point. The method is also applied to a real-world drug discovery problem, where the explanation vectors help identify the features that make certain compounds mutagenic or non-mutagenic. The results align with established knowledge about toxicophores and detoxicophores, demonstrating the effectiveness of the approach in capturing domain-specific insights. The paper discusses the limitations of the method, including cases where the derivative is zero, and the behavior of explanation vectors near the boundaries of the training data. It also addresses the assumption of stationarity in the data and the need for appropriate methods when dealing with non-stationary data. In conclusion, the proposed method provides a way to understand the decisions made by complex classifiers, enabling a deeper understanding of the underlying patterns and features that influence predictions. This approach is particularly useful in applications where interpretability is crucial, such as in drug discovery and other domains where the reasoning behind predictions needs to be transparent.