Pith. sign in

REVIEW 2 major objections 4 minor 80 references

The State of Post-Hoc Local XAI Techniques for Image Processing: Challenges and Motivations

T0 review · 2 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This survey argues that explainable AI's central obstacle is the absence of a shared formal definition of explainability, which leaves no way to determine whether one explanation method is better than another.

desk verdict A readable but derivative XAI survey whose main future-direction claim in §8.1 is directly contradicted by its own §6.4 discussion of multi-modal explanation works. read the letter →

arxiv 2501.06253 v1 pith:4J5DMKYV submitted 2025-01-09 cs.CV cs.AI

Rech Leong Tian Poh , Sye Loong Keoh , Liying Li This is my paper · ORCID
classification cs.CVcs.AI
keywords explainableAIpost-hocexplanationlocalmethodsimageprocessingdisagreementproblemevaluationtrustworthinesssurvey
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This is a survey of post-hoc local explanation techniques for image-processing models, meaning methods that explain a single prediction rather than the whole model. The paper sets out the reasons XAI matters, reviews representative techniques such as individual conditional expectation, counterfactual explanations, local surrogate models, and Shapley-value attribution, and then maps the obstacles that keep these methods from being trustworthy. Its central claim is that the field is blocked less by a shortage of techniques than by the lack of an agreed definition of explainability and of evaluation metrics that agree with each other. The paper also proposes future directions, including combining explanation modalities and involving human stakeholders and ground-truth considerations in evaluation.

What carries the argument

The analytical engine is a challenge taxonomy that ties four motivations for XAI, namely regulatory pressure, industrial adoption, technical advancement, and societal impact, to concrete obstacles such as missing formalism, interpretability of explanations, complexity-accuracy trade-offs, diversification of approaches, weak causal explanations, and limitations of current methods. The unresolved disagreement problem is the key mechanism: when multiple XAI algorithms or multiple evaluation metrics rank features differently for the same input, practitioners have no principled way to choose an explanation. The paper keeps returning to alignment-style metrics that measure agreement between explanations, and to the proposed combination of explanation modalities as the main lever for making outputs more human-understandable.

What would settle it

A literature search for published systems that combine visual saliency explanations with natural-language explanations for image classifiers would test the paper's core novelty claim; the paper itself cites examples of such systems, so if those count as multi-modal XAI, the claim that 'we have yet to see any works' in this direction is refuted.

Watch

Extended reading notes

Core claim

The paper's central claim is that without a satisfactory definition of explainability and interpretability, it is not possible to determine whether new XAI approaches are better at explaining machine-learning models, and therefore no single de-facto XAI approach has emerged. It identifies two disagreement problems that follow from this: different explanation algorithms can rank the same features differently, and different evaluation metrics can disagree about which explanation is most faithful. On the image-processing side, the paper reviews local post-hoc methods and their practical limitations, including computational cost, sensitivity to feature correlation, and lack of native support for object-detection models. As a way forward, it proposes intra-model and multi-modal XAI, combining explanations of the same type or pairing visual saliency with natural-language output, along with human-in-the-loop evaluation and a better mapping of stakeholder requirements to ground truth.

Load-bearing premise

The paper's proposed multi-modal future direction assumes that no published work yet combines multiple XAI output modalities, even though it cites multi-modal explanation systems earlier in the text, so that assumption is the most fragile part of the argument.

Editorial extensions

If this is right

  • If no agreed definition of explainability exists, new XAI approaches cannot be compared against a baseline, so research progress cannot be measured at all.
  • If explanation algorithms disagree on the same input, practitioners must rely on alignment metrics and benchmark leaderboards to decide which explanation to trust.
  • If evaluation metrics disagree about faithfulness, current benchmarks do not give end users enough guidance to select an XAI method.
  • If multi-modal and intra-model XAI succeed, visual saliency could be paired with natural-language explanations for image classifiers, improving interpretability without requiring users to read heatmaps alone.
  • If human-in-the-loop evaluation and stakeholder requirement analysis are adopted, regulatory 'right to explanation' mandates would have a concrete implementation path.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: if explainability cannot be formally defined, regulators cannot operationalize the 'right to explanation' without effectively choosing or mandating an evaluation metric, coupling AI regulation to XAI standardization.
  • Beyond the paper: a concrete experiment measuring multiple alignment and faithfulness metrics across several local explainers on a fixed set of images would quantify how often the disagreement problem occurs and how severe it is.
  • Beyond the paper: pairing saliency maps with vision-language captions could be tested directly in user studies that compare comprehension time and decision accuracy against visual-only explanations.
  • Beyond the paper: the disagreement problem is likely amplified for object-detection models, since the paper notes attribution methods need per-class tuning, and per-class explanations may disagree in ways that global metrics do not capture.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. This manuscript is a narrative survey of post-hoc local explainability for image-processing models. It begins by reviewing terminology (explainability, interpretability, trustworthiness, repeatability, reproducibility, stability, causality, cotenability, faithfulness), then summarizes motivations for XAI (regulatory, industrial, technical, societal) and four post-hoc local methods (ICE, counterfactual explanations, LIME, SHAP). The central body catalogues challenges (lack of formalism, interpretability of explanations, complexity–accuracy trade-offs, diversification of methods, causal explanations, limitations of current approaches, debugging) and open problems (the disagreement problem, evaluation metrics, disagreement among metrics). It closes with three future directions: multi-modal/intra-model XAI, enhanced evaluation metrics, and better understanding of ground truth and stakeholder requirements.

Significance. The paper's descriptive content is broadly consistent with the literature, and it offers a compact entry point to the XAI discourse. It is not an empirical contribution and does not provide a systematic or machine-checked methodology, but several of its organizing categories (motivations vs challenges, objective vs human-centric evaluation) are useful. The §6.1 point about the absence of a shared formal definition of explainability is plausible and connects to a genuine debate. However, the survey's reliability is weakened by at least one self-contradiction in its proposed future directions, and the recommendations will require substantive re-scoping rather than copy-editing alone.

major comments (2)
  1. [§8.1; cf. §6.4] The claim that multi-modal/intra-model XAI is an unoccupied direction is contradicted by the paper's own earlier discussion. Section 6.4 states that 'Multi-modal explanations have also been explored in [14]' and cites [59] (Park et al., Multimodal Explanations) as a system combining textual justification with visual pointing; [14] (REX) similarly pairs textual explanations with visual regions. Section 8.1 nevertheless asserts that 'at the current time of writing, we have yet to see any works that progressed in this direction.' Because proposing future directions is one of the three listed contributions (§1), this internal contradiction directly undermines the novelty claim. The passage should be re-scoped to a narrower gap (for example, combining SHAP/LIME-style attributions with vision-language models such as CLIP) or the proposal should be reframed as consolidation/extension rather than a new direction.
  2. [§7.1 and §7.2] The evaluation discussion is internally inconsistent about ground truth. Section 6.1 identifies 'the lack of ground truth explanations' as a major challenge, and §7.2 cites [10] for the absence of a foundational ground truth, yet §7.1 defines Feature Agreement, Rank Agreement, Sign Agreement, Signed Rank Agreement, Rank Correlation, and Pairwise Rank Agreement as comparisons against 'the corresponding ground truth explanation' without specifying how that ground truth is obtained. If these metrics are limited to synthetic datasets or manually constructed ground truths, that restriction must be stated; otherwise the reader cannot tell whether the open-problem section is describing a solvable benchmarking protocol or restating the foundational gap.
minor comments (4)
  1. [References] The reference list contains duplicate entries that should be merged: [1] and [2] are the same paper, [7] and [9] are the same paper in different forms, [22] and [23] are the same arXiv preprint, and [48] and [49] are the same book chapter.
  2. [Throughout] There are numerous typographical and spelling errors that should be corrected in a copy-editing pass: 'Explicable AI' in the Section 3 heading, 'Socio-Techinical' in §2, 'repeatablity' in §3.4, 'Shapely Values' in §5.4, 'Mutli-Modal' in §8.1, 'intepretability' in §6.4, and 'empirial' in §7.3.
  3. [Figures] Several figures are reproduced from external sources (e.g., Figures 1–5 from [30], [48], [44]) without source annotations in the captions; the manuscript should state permissions/reuse details, and Figures 7 and 8 should identify the benchmark and axes they illustrate.
  4. [General] Because the paper makes comparative frequency claims such as 'the majority of XAI works is mainly on image and textual data' (§6.6) and 'few works actually focused on a human-in-the-loop approach' (§8.2), a short methodology note describing the search process and inclusion criteria for the 83 cited papers would make those claims verifiable.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is a literature survey with no derivation chain, fitted parameters, or load-bearing self-citation; the §8.1 novelty inconsistency is a factual/correctness issue, not circularity.

full rationale

This manuscript is a survey/review of post-hoc local XAI techniques. It contains no mathematical derivations, no fitted parameters, no empirical pipeline, and no prediction that is constructed from its own inputs. Its central claims, such as the lack of a formal definition of explainability in §6.1, are arguments about the literature and are supported by external citations, not by a self-referential chain. I found no self-citations by the authors that are load-bearing, no uniqueness theorem imported from prior author work, and no ansatz smuggled in via citation. One notable issue is flagged for the record: §8.1 states that 'at the current time of writing, we have yet to see any works that progressed in this direction' for multi-modal/intra-model XAI, while §6.4 of the same paper states that 'Multi-modal explanations have also been explored in [14]' and cites [59], Park et al., 'Multimodal Explanations: Justifying Decisions and Pointing to the Evidence.' This is an internal inconsistency affecting the novelty of a stated future direction, and it should be corrected or scoped. However, this is a correctness and accuracy concern, not a circularity concern under the enumerated patterns: no derivation reduces to its inputs, and no claim is equivalent to its premise by construction. Accordingly, the circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper is a review; it introduces no free parameters or new entities. Its conclusions rely on the selection and interpretation of existing literature and on a subjective taxonomy.

assumptions (3)
  • domain assumption The categorization of XAI survey papers into Domain-Specific, Human-Centric, and Socio-Technical categories (Table 1) is meaningful and correctly applied.
    The paper's synthesis and interpretation of the literature rests on this taxonomy, but no explicit rubric or validation is provided.
  • domain assumption The selected references are representative of the XAI literature and are accurately summarized.
    The paper's claims about challenges, open problems, and future directions depend on the accuracy and completeness of its literature review.
  • domain assumption The term 'cotenability' is used in the sense of feature co-dependence, and this concept is relevant to evaluating explanations.
    The paper introduces 'cotenability' without a formal definition or citation, yet uses it to argue for causal explanations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The State of Post-Hoc Local XAI Techniques for Image Processing: Challenges and Motivations." pith.science (2026). https://pith.science/paper/4J5DMKYV

@misc{pith2026250106253,
  author       = {Pith},
  title        = {Pith review of: The State of Post-Hoc Local XAI Techniques for Image Processing: Challenges and Motivations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4J5DMKYV}},
  note         = {Machine review of arXiv:2501.06253}
}
read the original abstract

As complex AI systems further prove to be an integral part of our lives, a persistent and critical problem is the underlying black-box nature of such products and systems. In pursuit of productivity enhancements, one must not forget the need for various technology to boost the overall trustworthiness of such AI systems. One example, which is studied extensively in this work, is the domain of Explainable Artificial Intelligence (XAI). Research works in this scope are centred around the objective of making AI systems more transparent and interpretable, to further boost reliability and trust in using them. In this work, we discuss the various motivation for XAI and its approaches, the underlying challenges that XAI faces, and some open problems that we believe deserve further efforts to look into. We also provide a brief discussion of various XAI approaches for image processing, and finally discuss some future directions, to hopefully express and motivate the positive development of the XAI research space.

Figures

Figures reproduced from arXiv: 2501.06253 by the authors.

Figure 1
Figure 1. ICE: a) Understanding why the purple point is assigned to the negative class by considering what would happen if either [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
Figure 2
Figure 2. Counterfactual Explanations: a) Find out the explanation for which the data point (purple point 1) was classified negatively to [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Left: Image of a bowl of bread. Middle and right: LIME explanations for the top 2 classes (bagel, strawberry) for image [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Illustration of Shapley Values [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Illustration of SHAP Explanation [44] respectively. By observing the output explanation, one can derive that the higher concentrations of magenta spots on an MNIST digit represents a stronger correlation and argument that the model predicted it as such. For example, in…
Figure 6
Figure 6. Figure 6: A Summary of the Motivations & Challenges in XAI [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: OpenXAI Leaderboard Snippet [3] While the evaluation metrics and benchmarking approaches served their purpose, it is not always practical to make assumptions during the evaluation of ground truth explanations. For example, the authors of [10] have identified the Manusc…
Figure 8
Figure 8. Figure 8: ConsisXAI Overview [10] The main pipeline and stages are broken down as follows: • Analysis of a given dataset to identify a set of features that are considered indispensable for the prediction process (ground truth extraction). • Extracting the features that are actua…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

80 extracted references · 29 canonical work pages

  1. [14]

    Shi Chen and Qi Zhao. 2022. REX: Reasoning-aware and Grounded Explanation. arXiv:2203.06107 [cs.CV]

    work page Pith review arXiv 2022

  2. [59]

    Dong Huk Park, Lisa Anne Hendricks, Zeynep Akata, Anna Rohrbach, Bernt Schiele, Trevor Darrell, and Marcus Rohrbach. 2018. Multimodal Explanations: Justifying Decisions and Pointing to the Evidence. arXiv:1802.08129 [cs.AI]

    arXiv 2018

  3. [2]

    Amina Adadi and Mohammed Berrada. 2018. Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI). IEEE Access 6 (2018), 52138–52160. https://doi.org/10.1109/ACCESS.2018.2870052

    arXiv 2018

  4. [10]

    Mohamed Karim Belaid, Eyke Hüllermeier, Maximilian Rabus, and Ralf Krestel. 2022. Do We Need Another Explainable AI Method? Toward Unifying Post-hoc XAI Evaluation Methods into an Interactive and Multi-dimensional Benchmark. http://arxiv.org/abs/2207.14160 arXiv:2207.14160 [cs]

    work page Pith review arXiv 2022

  5. [3]

    Chirag Agarwal, Satyapriya Krishna, Eshika Saxena, Martin Pawelczyk, Nari Johnson, Isha Puri, Marinka Zitnik, and Himabindu Lakkaraju. 2023. OpenXAI: Towards a Transparent Evaluation of Model Explanations. http://arxiv.org/abs/2206.11104 arXiv:2206.11104 [cs]

    arXiv 2023

  6. [4]

    Muhammad Aurangzeb Ahmad, Ankur Teredesai, and Carly Eckert. 2018. Interpretable Machine Learning in Healthcare. In 2018 IEEE International Conference on Healthcare Informatics (ICHI) . IEEE Computer Society, Los Alamitos, CA, USA, 447–447. https://doi.org/10.1109/ICHI.2018.00095

    arXiv 2018

  7. [5]

    What is relevant in a text document?

    Leila Arras, Franziska Horn, Grégoire Montavon, Klaus-Robert Müller, and Wojciech Samek. 2017. "What is relevant in a text document?": An interpretable machine learning approach. PLOS ONE 12, 8 (08 2017), 1–23. https://doi.org/10.1371/journal.pone.0181142

    doi:10.1371/journal.pone.0181142 2017

  8. [7]

    Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador García, Sergio Gil-López, Daniel Molina, Richard Benjamins, Raja Chatila, and Francisco Herrera. 2019. Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI. arXiv:1910.100...

    arXiv 2019

Show all 80 references
  1. [8]

    Brian Barr, Noah Fatsi, Leif Hancox-Li, Peter Richter, Daniel Proano, and Caleb Mok. 2023. The Disagreement Problem in Faithfulness Metrics. http://arxiv.org/abs/2311.07763 arXiv:2311.07763 [cs]

    2023 arXiv

  2. [9]

    Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador Garcia, Sergio Gil-Lopez, Daniel Molina, Richard Benjamins, Raja Chatila, and Francisco Herrera. 2020. Explainable Artificial Intelligence (XAI): Concepts...

    2020 doi

  3. [11]

    Florian Bordes, Richard Yuanzhe Pang, Anurag Ajay, Alexander C. Li, Adrien Bardes, Suzanne Petryk, Oscar Mañas, Zhiqiu Lin, Anas Mahmoud, Bargav Jayaraman, Mark Ibrahim, Melissa Hall, Yunyang Xiong, Jonathan Lebensold, Candace Ross, Srihari Jayakumar, Chuan Guo, Diane Boucha- ...

    2024 arXiv

  4. [12]

    Nadia Burkart and Marco F. Huber. 2020. A Survey on the Explainability of Supervised Machine Learning. arXiv:2011.07876 [cs.LG]

    2020 arXiv

  5. [13]

    Carvalho, Eduardo M

    Diogo V. Carvalho, Eduardo M. Pereira, and Jaime S. Cardoso. 2019. Machine Learning Interpretability: A Survey on Methods and Metrics.Electronics 8, 8 (2019). https://doi.org/10.3390/electronics8080832

    2019 doi

  6. [15]

    Roberto Confalonieri, Ludovik Coba, Benedikt Wagner, and Tarek R. Besold. 2021. A historical perspective of explainable Ar- tificial Intelligence. WIREs Data Mining and Knowledge Discovery 11, 1 (2021), e1391. https://doi.org/10.1002/widm.1391 arXiv:https://wires.onlinelibrary...

    2021 doi

  7. [16]

    Council of European Union. 2017. Council regulation (EU) no 269/2017. http://ec.europa.eu/justice/dataprotection/reform/files/regulation_oj_en.pdf

    2017

  8. [17]

    Susanne Dandl, Christoph Molnar, Martin Binder, and Bernd Bischl. 2020. Multi-Objective Counterfactual Explanations . Springer International Publishing, 448–469. https://doi.org/10.1007/978-3-030-58112-1_31

    2020 doi

  9. [18]

    Marina Danilevsky, Kun Qian, Ranit Aharonov, Yannis Katsis, Ban Kawas, and Prithviraj Sen. 2020. A Survey of the State of Explainable AI for Natural Language Processing. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Lingu...

    2020

  10. [19]

    Jeffrey Dastin. 2018. Amazon scraps secret AI recruiting tool that showed bias against women. https://www.reuters.com/article/us-amazon-com- jobs-automation-insight-idUSKCN1MK08G

    2018

  11. [20]

    Richard Dazeley, Peter Vamplew, and Francisco Cruz. 2021. Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework and Survey. arXiv:2108.09003 [cs.AI]

    2021 arXiv

  12. [21]

    Derek Doran, Sarah Schulz, and Tarek R. Besold. 2017. What Does Explainable AI Really Mean? A New Conceptualization of Perspectives. CoRR abs/1710.00794 (2017). arXiv:1710.00794 http://arxiv.org/abs/1710.00794

    2017 arXiv

  13. [23]

    Finale Doshi-Velez and Been Kim. 2017. Towards A Rigorous Science of Interpretable Machine Learning. arXiv:1702.08608 [stat.ML]

    2017 arXiv

  14. [24]

    Vera Liao, Michael Muller, Mark O

    Upol Ehsan, Q. Vera Liao, Michael Muller, Mark O. Riedl, and Justin D. Weisz. 2021. Expanding Explainability: Towards Social Transparency in AI Systems. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for...

    2021

  15. [25]

    Upol Ehsan and Mark Riedl. 2020. Human-Centered Explainable AI: Towards a Reflective Sociotechnical Approach . Springer-Verlag, Berlin, Heidelberg, 449–466. https://doi.org/10.1007/978-3-030-60117-1_33

    2020 doi

  16. [26]

    Ghada Elkhawaga, Omar Elzeki, Mervat Abuelkheir, and Manfred Reichert. 2023. Evaluating Explainable Artificial Intelligence Methods Based on Feature Elimination: A Functionality-Grounded Approach. Electronics 12, 7 (March 2023), 1670. https://doi.org/10.3390/electronics12071670

    2023 doi

  17. [27]

    European Commission. 2020. Ethic Guidelines for Trustworthy AI. https://ec.europa.eu/futurium/en/ai-alliance-consultation

    2020

  18. [28]

    Europeon Commission. 2020. The Royal Society: Explainable AI: the basics. https://royalsociety.org/-/media/policy/projects/explainable-ai/AI-and- interpretability-policy-briefing.pdf

    2020

  19. [29]

    Krishna Gade, Sahin Cem Geyik, Krishnaram Kenthapadi, Varun Mithal, and Ankur Taly. 2020. Explainable AI in Industry: Practical Challenges and Lessons Learned: Implications Tutorial. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (Barcelona...

    2020

  20. [30]

    Alex Goldstein, Adam Kapelner, Justin Bleich, and Emil Pitkin. 2014. Peeking Inside the Black Box: Visualizing Statistical Learning with Plots of Individual Conditional Expectation. arXiv:1309.6392 [stat.AP]

    2014 arXiv

  21. [31]

    Right to Explanation

    Bryce Goodman and Seth Flaxman. 2017. European Union Regulations on Algorithmic Decision Making and a “Right to Explanation”. AI Magazine 38, 3 (2017), 50–57. https://doi.org/10.1609/aimag.v38i3.2741 arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1609/aimag.v38i3.2741

    2017 doi

  22. [32]

    Rory Mc Grath, Luca Costabello, Chan Le Van, Paul Sweeney, Farbod Kamiab, Zhao Shen, and Freddy Lecue. 2018. Interpretable Credit Application Predictions With Counterfactual Explanations. arXiv:1811.05245 [cs.AI]

    2018 arXiv

  23. [33]

    Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. 2018. A Survey of Methods for Explaining Black Box Models. ACM Comput. Surv. 51, 5, Article 93 (aug 2018), 42 pages. https://doi.org/10.1145/3236009

    2018 doi

  24. [34]

    Anna Hedström, Leander Weber, Dilyara Bareeva, Daniel Krakowczyk, Franz Motzkus, Wojciech Samek, Sebastian Lapuschkin, and Marina M.-C. Höhne. 2023. Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyond. http: //arxiv.org/abs/2...

    2023 arXiv

  25. [35]

    Andreas Holzinger, Georg Langs, Helmut Denk, Kurt Zatloukal, and Heimo Müller. 2019. Causability and explainabilty of artificial intelligence in medicine. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 9 (04 2019), e1312. https://doi.org/10.1002/widm.1312...

    2019 doi

  26. [36]

    Nirmal Sobha Kartha. 2021. Explain yourself - A Primer on ML Interpretability & Explainability. https://thegradient.pub/explain-yourself/

    2021

  27. [37]

    Satyapriya Krishna, Tessa Han, Alex Gu, Javin Pombra, Shahin Jabbari, Steven Wu, and Himabindu Lakkaraju. 2022. The Disagreement Problem in Explainable Machine Learning: A Practitioner’s Perspective. arXiv:2202.01602 [cs.LG]

    2022 arXiv

  28. [38]

    David Krueger, Ethan Caballero, Joern-Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. 2021. Out-of-Distribution Generalization via Risk Extrapolation (REx). arXiv:2003.00688 [cs.LG]

    2021 arXiv

  29. [39]

    Markus Langer, Daniel Oster, Timo Speith, Holger Hermanns, Lena Kästner, Eva Schmidt, Andreas Sesing, and Kevin Baum. 2021. What Do We Want From Explainable Artificial Intelligence (XAI)? - A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI R...

    2021 arXiv

  30. [40]

    Phuong Quynh Le, Meike Nauta, Van Bach Nguyen, Shreyasi Pathak, Jörg Schlötterer, and Christin Seifert. 2023. Benchmarking eXplainable AI - A Survey on Available Toolkits and Open Challenges. InProceedings of the Thirty-Second International Joint Conference on Artificial Intel...

    2023 doi

  31. [41]

    Zachary C. Lipton. 2018. The Mythos of Model Interpretability: In Machine Learning, the Concept of Interpretability is Both Important and Slippery. Queue 16, 3 (jun 2018), 31–57. https://doi.org/10.1145/3236386.3241340

    2018

  32. [42]

    Luca Longo, Randy Goebel, Freddy Lecue, Peter Kieseberg, and Andreas Holzinger. 2020. Explainable Artificial Intelligence: Concepts, Applications, Research Challenges and Visions. In 4th International Cross-Domain Conference for Machine Learning and Knowledge Extraction (CD-MA...

    2020 doi

  33. [43]

    Adriano Lucieri, Muhammad Naseer Bajwa, Andreas Dengel, and Sheraz Ahmed. 2020. Achievements and Challenges in Explaining Deep Learning based Computer-Aided Diagnosis Systems. arXiv:2011.13169 [cs.AI]

    2020 arXiv

  34. [44]

    Scott Lundberg. 2018. Multi-input Gradient Explainer MNIST Example. https://shap.readthedocs.io

    2018

  35. [45]

    Lundberg and Su-In Lee

    Scott M. Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. CoRR abs/1705.07874 (2017). arXiv:1705.07874 http://arxiv.org/abs/1705.07874

    2017 arXiv

  36. [46]

    Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2019. A Survey on Bias and Fairness in Machine Learning. arXiv:1908.09635 [cs.LG]

    2019 arXiv

  37. [47]

    AlShami, and Jugal Kalita

    Melkamu Mersha, Khang Lam, Joseph Wood, Ali K. AlShami, and Jugal Kalita. 2024. Explainable artificial intelligence: A survey of needs, techniques, applications, and future direction. Neurocomputing 599 (2024), 128111. https://doi.org/10.1016/j.neucom.2024.128111

    2024

  38. [48]

    Christoph Molnar. 2022. Interpretable Machine Learning (2 ed.). https://christophm.github.io/interpretable-ml-book

    2022

  39. [49]

    Christoph Molnar. 2022. Interpretable machine learning. https://christophm.github.io/interpretable-ml-book/shapley.html

    2022

  40. [50]

    Christoph Molnar, Giuseppe Casalicchio, and Bernd Bischl. 2020. Interpretable Machine Learning – A Brief History, State-of-the-Art and Challenges . Springer International Publishing, Cham, 417–431. https://doi.org/10.1007/978-3-030-65965-3_28

    2020 doi

  41. [51]

    Milad Moradi, Ke Yan, David Colwell, Matthias Samwald, and Rhona Asgari. 2023. Model-agnostic explainable artificial intelligence for object detection in image data. arXiv:2303.17249 [cs.CV]

    2023 arXiv

  42. [52]

    Raha Moraffah, Mansooreh Karami, Ruocheng Guo, Adrienne Raglin, and Huan Liu. 2020. Causal Interpretability for Machine Learning - Problems, Methods and Evaluation. SIGKDD Explor. Newsl. 22, 1 (may 2020), 18–33. https://doi.org/10.1145/3400051.3400058

    2020

  43. [53]

    Mueller, Robert R

    Shane T. Mueller, Robert R. Hoffman, William Clancey, Abigail Emrey, and Gary Klein. 2019. Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI. arXiv:1902.01876 [cs.AI]

    2019 arXiv

  44. [54]

    James Murdoch, Chandan Singh, Karl Kumbier, Reza Abbasi-Asl, and Bin Yu

    W. James Murdoch, Chandan Singh, Karl Kumbier, Reza Abbasi-Asl, and Bin Yu. 2019. Definitions, methods, and applications in interpretable machine learning. Proceedings of the National Academy of Sciences 116, 44 (2019), 22071–22080. https://doi.org/10.1073/pnas.1900654116 arXi...

    2019 doi

  45. [55]

    Mohammad Nagahisarchoghaei, Nasheen Nur, Logan Cummins, Nashtarin Nur, Mirhossein Mousavi Karimi, Shreya Nandanwar, Siddhartha Bhattacharyya, and Shahram Rahimi. 2023. An Empirical Survey on Explainable AI Technologies: Recent Trends, Use-Cases, and Categories from Technical a...

    2023 doi

  46. [56]

    Mohammad Naiseh, Nan Jiang, Jianbing Ma, and Raian Ali. 2020. Personalising Explainable Recommendations: Literature and Conceptualisation . Springer International Publishing, Cham, 518–533. https://doi.org/10.1007/978-3-030-45691-7_49

    2020 doi

  47. [57]

    Anh Nguyen, Jason Yosinski, and Jeff Clune. 2015. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 427–436. https://doi.org/10.1109/CVPR.2015.7298640

    2015

  48. [58]

    Osoba and William Welser IV

    Osonde A. Osoba and William Welser IV. 2017. An Intelligence in Our Image: The Risks of Bias and Errors in Artificial Intelligence . RAND Corporation, Santa Monica, CA. https://doi.org/10.7249/RR1744

    2017 doi

  49. [60]

    Milda Poceviči¯ut˙e, Gabriel Eilertsen, and Claes Lundström. 2020. Survey of XAI in Digital Pathology . Springer International Publishing, 56–88. https://doi.org/10.1007/978-3-030-50402-1_4

    2020 doi

  50. [61]

    Eleonora Poeta, Gabriele Ciravegna, Eliana Pastor, Tania Cerquitelli, and Elena Baralis. 2023. Concept-based Explainable Artificial Intelligence: A Survey. arXiv:2312.12936 [cs.AI] https://arxiv.org/abs/2312.12936

    2023

  51. [62]

    Jain R. 2024. Transparency in AI Decision Making: A Survey of Explainable AI Methods and Applications. ART 2, 1 (Jan. 2024), 1–10. https: //doi.org/10.23880/art-16000110 Manuscript submitted to ACM 24 Rech Leong Tian Poh, Sye Loong Keoh, and Liying Li

    2024 doi

  52. [63]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. arXiv:2103.000...

    2021 arXiv

  53. [64]

    Gabrielle Ras, Marcel van Gerven, and Pim Haselager. 2018. Explanation Methods in Deep Learning: Users, Values, Concerns and Challenges. arXiv:1803.07517 [cs.AI]

    2018 arXiv

  54. [65]

    Gabrielle Ras, Ning Xie, Marcel van Gerven, and Derek Doran. 2021. Explainable Deep Learning: A Field Guide for the Uninitiated. arXiv:2004.14545 [cs.LG]

    2021 arXiv

  55. [66]

    Silva, Fried-Michael Dahlweid, Hendrik von Tengg-Kobligk, Ronald M

    Mauricio Reyes, Raphael Meier, Sérgio Pereira, Carlos A. Silva, Fried-Michael Dahlweid, Hendrik von Tengg-Kobligk, Ronald M. Summers, and Roland Wiest. 2020. On the Interpretability of Artificial Intelligence in Radiology: Challenges and Opportunities. Radiology: Artificial In...

    2020 doi

  56. [67]

    Why Should I Trust You?

    Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why Should I Trust You?": Explaining the Predictions of Any Classifier. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/2939672.2939778

    2016

  57. [68]

    Margarita Robles Carrillo. 2020. Artificial intelligence: From ethics to law. Telecommunications Policy 44, 6 (2020), 101937. https://doi.org/10.1016/j. telpol.2020.101937 Artificial intelligence, economy and society

    2020

  58. [69]

    Thomas Rojat, Raphaël Puget, David Filliat, Javier Del Ser, Rodolphe Gelin, and Natalia Díaz-Rodríguez. 2021. Explainable Artificial Intelligence (XAI) on TimeSeries Data: A Survey. arXiv:2104.00950 [cs.LG]

    2021 arXiv

  59. [70]

    Cynthia Rudin. 2019. Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead. arXiv:1811.10154 [stat.ML]

    2019 arXiv

  60. [71]

    Waddah Saeed and Christian Omlin. 2023. Explainable AI (XAI): A systematic meta-survey of current challenges and future opportunities. Knowledge-Based Systems 263 (March 2023), 110273. https://doi.org/10.1016/j.knosys.2023.110273

    2023

  61. [72]

    Wojciech Samek and Klaus-Robert Müller. 2019. Towards Explainable Artificial Intelligence . Springer International Publishing, 5–22. https: //doi.org/10.1007/978-3-030-28954-6_1

    2019 doi

  62. [73]

    Wojciech Samek, Thomas Wiegand, and Klaus-Robert Müller. 2017. Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models. arXiv:1708.08296 [cs.AI]

    2017 arXiv

  63. [74]

    Paul Schwerdtner, Florens Greßner, Nikhil Kapoor, Felix Assion, René Sass, Wiebke Günther, Fabian Hüger, and Peter Schlicht. 2020. Risk Assessment for Machine Learning Models. arXiv:2011.04328 [cs.LG]

    2020 arXiv

  64. [75]

    Weld, and Leah Findlater

    Alison Smith-Renner, Ron Fan, Melissa Birchfield, Tongshuang Wu, Jordan Boyd-Graber, Daniel S. Weld, and Leah Findlater. 2020. No Explainability without Accountability: An Empirical Study of Explanations and Feedback in Interactive ML. Association for Computing Machinery, New ...

    2020

  65. [76]

    Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2014. Intriguing properties of neural networks. arXiv:1312.6199 [cs.CV]

    2014 arXiv

  66. [77]

    The European Commission. 2021. The AI Act. https://artificialintelligenceact.eu/the-act/

    2021

  67. [78]

    Bissyandé, and Jacques Klein

    Lisa Veiber, Kevin Allix, Yusuf Arslan, Tegawendé F. Bissyandé, and Jacques Klein. 2020. Challenges Towards Production-Ready Explainable Machine Learning. In 2020 USENIX Conference on Operational Machine Learning (OpML 20) . USENIX Association. https://www.usenix.org/conferenc...

    2020

  68. [79]

    Scott Vrieze. 2012. Model Selection and Psychological Theory: A Discussion of the Differences Between the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC). Psychological methods 17 (02 2012), 228–43. https://doi.org/10.1037/a0027127

    2012 doi

  69. [80]

    Sandra Wachter, Brent Mittelstadt, and Chris Russell. 2018. Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR. arXiv:1711.00399 [cs.AI]

    2018 arXiv

  70. [81]

    Hao Yuan, Haiyang Yu, Shurui Gui, and Shuiwang Ji. 2023. Explainability in Graph Neural Networks: A Taxonomic Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 5 (2023), 5782–5799. https://doi.org/10.1109/TPAMI.2022.3204236

    2023

  71. [82]

    Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du. 2023. Explainability for Large Language Models: A Survey. arXiv:2309.01029 [cs.CL] https://arxiv.org/abs/2309.01029

    2023 arXiv

  72. [83]

    Jianlong Zhou, Amir H Gandomi, Fang Chen, and Andreas Holzinger. 2021. Evaluating the quality of machine learning explanations: A survey on methods and metrics. Electronics 10, 5 (2021), 593. Manuscript submitted to ACM

    2021

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.