From Flexibility to Manipulation: The Slippery Slope of XAI Evaluation
| Autor: | Wickstrøm, Kristoffer, Höhne, Marina Marie-Claire, Hedström, Anna |
|---|---|
| Rok vydání: | 2024 |
| Předmět: | |
| Druh dokumentu: | Working Paper |
| Popis: | The lack of ground truth explanation labels is a fundamental challenge for quantitative evaluation in explainable artificial intelligence (XAI). This challenge becomes especially problematic when evaluation methods have numerous hyperparameters that must be specified by the user, as there is no ground truth to determine an optimal hyperparameter selection. It is typically not feasible to do an exhaustive search of hyperparameters so researchers typically make a normative choice based on similar studies in the literature, which provides great flexibility for the user. In this work, we illustrate how this flexibility can be exploited to manipulate the evaluation outcome. We frame this manipulation as an adversarial attack on the evaluation where seemingly innocent changes in hyperparameter setting significantly influence the evaluation outcome. We demonstrate the effectiveness of our manipulation across several datasets with large changes in evaluation outcomes across several explanation methods and models. Lastly, we propose a mitigation strategy based on ranking across hyperparameters that aims to provide robustness towards such manipulation. This work highlights the difficulty of conducting reliable XAI evaluation and emphasizes the importance of a holistic and transparent approach to evaluation in XAI. Comment: Published in ECCV 2024 Workshop on Explainable Computer Vision: Where are We and Where are We Going? Shorter non-archival version also appeared in the NeurIPS 2024 Interpretable AI workshop. Code is available at \url{https://github.com/Wickstrom/quantitative-xai-manipulation} |
| Databáze: | arXiv |
| Externí odkaz: |
|