Robustness-TNNLS

Post-hoc interpretability methods play a critical role in explainable artificial intelligence (XAI), as they pinpoint portions of data that a trained deep learning model deemed important to make a decision. However, different post-hoc interpretability methods often provide different results, casting doubts on their reliability. For this reason, several evaluation strategies have been proposed in the literature to validate the accuracy of post-hoc interpretability. Yet, these evaluation strategies provide an average assessment. A critical evaluation metric that is of paramount importance in several fields, from medicine, to engineering and the natural sciences is largely missing: the uncertainty of interpretability results. Uncertainty is crucial to understand how post-hoc interpretability methods perform across a given dataset and it should be provided along with the average performance, as commonly done in other scientific sectors. We propose an approach and two new quantitative metrics to measure the uncertainty, also referred to as robustness, of post-hoc interpretability methods. We show that the robustness is generally linked to the average performance of post-hoc interpretability methods (albeit not always): the better the average performance, the more robust the method.