Pith. sign in

REVIEW 4 major objections 6 minor 110 references

Your Model Is Unfair, Are You Even Aware? Inverse Relationship Between Comprehension and Trust in Explainability Visualizations of Biased ML Models

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that explainability visualizations that raise comprehension of a biased ML model reduce trust in it, because comprehension amplifies perceived bias, and perceived bias drives distrust.

desk verdict Large, honest HCI study with a provocative inverse comprehension-trust finding; the causal mediation claim is plausible but not fully nailed by Experiment 3. read the letter →

arxiv 2508.00140 v1 pith:P7GCONNH submitted 2025-07-31 cs.HC cs.AI

Zhanna Kaufman , Madeline Endres , Cindy Xiong Bearfield , Yuriy Brun This is my paper · ORCID
classification cs.HCcs.AI
keywords explainabilityvisualizationmachinelearningbiasusertrustcomprehensionperceptiontaxonomycausalmediationcrowdsourcedstudy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper studies how local explainability visualizations of a biased machine-learning model affect non-expert users' comprehension, perception of bias, and trust. Using a taxonomy of 54 design characteristics abstracted from 26 visualization tools, it runs user studies with 818 participants on six visualizations (SHAP waterfall and force plots, LIME, ELI5, Ceteris-Paribus profiles, and Anchors). Its central finding is an inverse relationship: the better users understand the model, the less they trust it, and this is explained, and causally confirmed, by bias perception acting as a near-complete mediator. The paper's case is that visualization design can independently move comprehension, bias perception, and trust, so designers can reveal bias and maintain trust only by addressing the model's actual fairness.

What carries the argument

The load-bearing instrument is a taxonomy of 54 visualization design characteristics across 27 dimensions (e.g., whether impact direction is shown by color, bar, position, or printed value; whether feature values are binned; whether alternative inputs are displayed), built from 26 existing local-explanation visualizations. The taxonomy lets the authors abstract real tools into manipulable design variables and then test them in controlled experiments. The second mechanism is the mediation model that traces comprehension through bias perception to trust, with an indirect effect of $-0.63$ and a near-zero direct effect.

What would settle it

A pre-registered study that re-measures comprehension by asking participants to predict which class the model will output for new inputs (rather than to identify which features matter most), across the same six visualizations; if the comprehension-trust correlation is no longer negative, the claim that comprehension itself drives distrust is undermined.

Watch

Extended reading notes

Core claim

In a series of user studies with 818 non-expert participants viewing loan-recommendation explanations from five state-of-the-art tools, the paper finds that comprehension and trust are negatively correlated ($r = -0.28$, $p < 0.001$). A mediation model shows the mechanism: comprehension positively predicts bias perception ($b = 0.27$, $p < 0.001$), bias perception negatively predicts trust ($b = -2.35$, $p < 0.001$), and the direct comprehension-to-trust effect is small and non-significant ($b = 0.05$, $p = 0.49$). Three controlled experiments support causality: making feature impacts explicit raises comprehension and bias perception while lowering trust; making the underlying model fairer lowers perceived bias and raises trust; and reducing perceived bias purely through visualization design raises trust ($d = 0.20$) even though comprehension changes negligibly ($d = -0.07$). The paper concludes that for biased models, better explanations make bias visible, and visible bias costs trust.

Load-bearing premise

The central mediation result rests on measuring all six visualizations with the same comprehension questions centered on feature-impact attribution; the paper concedes that CP profiles do not encode feature importance by design, so part of the cross-tool inverse relationship may be an artifact of this common yardstick.

Editorial extensions

If this is right

  • Visualization design that makes a biased model's feature impacts explicit will systematically depress user trust in the model, because explicitness increases bias perception.
  • Trust can be raised without changing the model by lowering bias perception through design (e.g., adding alternative-output plots, removing axes), even when comprehension stays about the same.
  • Comprehension and trust are not in inherent tension for fair models: with a fair model, high comprehension increases trust via lower bias perception.
  • Designers who want both comprehension and trust must either reduce the model's actual bias or carefully control which features are salient, because comprehension and bias perception move together.
  • Metrics for explainability should separate comprehension from perceived bias, since they often move in opposite directions for biased models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If this inverse relationship generalizes, then audits and regulatory disclosures that force clearer explanations of biased models may produce an apparent 'trust collapse' that is actually appropriate calibration; policy should interpret distrust of biased models as a feature, not a failure.
  • A testable extension: use a comprehension measure based on users' ability to predict model outputs rather than to state feature importance; the cross-tool negative correlation may shrink, while the individual-level mediation should persist.
  • The same design levers might be used to calibrate trust in other high-stakes domains (hiring, medical triage), where hiding bias behind complex visualizations currently inflates trust; this would need direct replication.
  • A follow-up could vary the protected attribute (e.g., age, race instead of sex) to test whether the salience of the biased feature, not just its visibility, drives the effect.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper asks how the design of local ML explainability visualizations affects non-experts' comprehension, perceived bias, and trust. The authors build a 27-dimension taxonomy from 26 visualizations, then run 11 Prolific surveys with 818 participants using a biased loan model, six state-of-the-art visualization tools (plus modified variants), and three follow-up experiments intended to establish causality. The central finding is a negative correlation between comprehension and trust, reportedly mediated by bias perception, with the interpretation that better explanations of biased models reduce trust by increasing perceived bias, and that visualization design can increase trust by lowering perceived bias even when actual model behavior is unchanged.

Significance. The research question is well chosen and the empirical investment is substantial: 818 participants, effect sizes for main effects, open stimuli and data in supplementary materials, and a reusable taxonomy of visualization design characteristics. If the mediation claim were firmly established, the paper would make an important contribution to explainable AI and visualization design. However, the strongest causal link—that perceived bias mediates the comprehension–trust relationship—currently rests on a confounded design change in Experiment 3 and on mediation evidence whose indirect effect is not formally tested. The significance of the work is therefore real but conditional on resolving these load-bearing concerns.

major comments (4)
  1. [Section 7.3 (Experiment 3)] The two conditions in Experiment 3 differ on several visualization dimensions at once: the 'lower bias perception' version adds CP plots on feature mouse-over, adds alternative outputs (D22), and increases visual complexity, while the 'higher bias perception' version removes the x/y axes to increase simplicity. The observed trust difference (d=0.20) can therefore be caused by any of these design changes rather than specifically by the intended mental state of perceived bias. Moreover, no mediation model is fitted to the Experiment 3 data; condition-level differences in bias perception and trust do not demonstrate that trust changed through perceived bias. Please either redesign the manipulation to vary a single taxonomy dimension (or hold complexity constant) and fit a formal mediation/causal-mediation analysis, or soften the causal language in the abstract and Section 7.
  2. [Section 7.2 (Experiment 2)] The fair model is described as a random forest model trained with the Seldonian toolkit, whereas the original biased model in Section 5.2 is a LightGBM classifier. If the two conditions differ in model algorithm as well as in fairness, then the reported trust increase (d=0.44) may partly reflect model type or presentation rather than fairness alone. Please state explicitly whether the biased comparator in Experiment 2 is the same algorithm; if not, rerun with the identical architecture or justify why the algorithm difference does not affect the fairness conclusion.
  3. [Sections 6 and 8 (comprehension metric)] The comprehension items C1–C3 all ask about feature importance and prediction direction or magnitude, but CP profiles do not encode feature importance by design, as the Limitations section itself concedes. The finding that CP has the lowest comprehension and highest trust may therefore be partly a measurement artifact. Because this tool-level pattern contributes to the aggregate negative correlation and to the mediation result, the paper should re-run the RQ2 analyses with a comprehension measure that is fair across explanation types (or with CP excluded) and report how the mediation estimates change.
  4. [Section 6 (mediation significance)] The mediation model reports the path coefficients and the point estimate of the indirect effect (b=-0.63), but no confidence interval or significance test for the indirect effect (e.g., bootstrap or Monte Carlo) is provided. The abstract's statement 'p < 0.001 for all' is not supported for the indirect effect. Please add an inferential test of the mediated effect, such as bootstrap confidence intervals, before claiming strong mediation.
minor comments (6)
  1. [Section 4 (RQ1 Summary)] The RQ1 Summary says 44 characteristics, while Section 4 and Figure 2 report 54 characteristics across 27 dimensions; please correct the discrepancy.
  2. [Section 5.2.2] The text says each comparison is 'identical, other than a single visualization characteristic or model property,' but Experiment 3 changes multiple characteristics; revise the general claim or describe Experiment 3 as varying a bundle of design dimensions.
  3. [Figure 4 caption] Figure 4 uses a weather/coat example while Section 5.2 describes loan scenarios; clarify in the caption that Figure 4 shows the tutorial model rather than the study stimuli.
  4. [Section 7.2] There is a typo in 'way that vas visible'; it should be 'way that was visible'.
  5. [Reference [35]] Reference [35] contains a corrupted URL ('tinyurl.com/2zaj8amsÆŠ'); please repair it.
  6. [Section 9] The statement that 'all 19 non-binary participants across survey variations found both fair and biased models to be discriminatory' is anecdotal and not backed by statistical analysis; either analyze this subgroup formally or present the observation as qualitative.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the taxonomy, measures, and manipulations are independent of the reported outcomes.

full rationale

The paper's derivation chain is empirical rather than definitional. The taxonomy (Section 4) is a fixed coding scheme developed from visual inspection of 26 visualizations and applied before outcome measurement; it is not fitted to the comprehension, bias-perception, or trust results. The central mediation result (Figure 5) estimates relationships among separately operationalized constructs: comprehension is measured by C1-C3 (feature-impact understanding), bias perception by B1-B4 (fairness judgments), and trust by T1-T5 (accuracy and willingness to rely). These are distinct measurements, so the negative comprehension-trust correlation and the mediation by bias perception are empirical findings, not artifacts of overlapping definitions. The RQ3 experiments manipulate objective visualization features or model fairness independently of the measured outcomes: Experiment 1 adds explicit mouse-over values to CP plots, Experiment 2 swaps an unfair model for a Seldonian-constrained fair model, and Experiment 3 changes design characteristics (adding CP plots on mouse-over, removing axes). None of these manipulations is defined in terms of the outcome variables, so the observed effects are not forced by construction. The Section 8 limitation that CP profiles do not encode feature importance by design is a measurement-validity caveat, not a circular reduction; the authors explicitly concede that alternative comprehension questions might capture CP’s intended information. Self-citations (e.g., [28] for visualization effects on trust and perceived bias) provide background and motivation but are not load-bearing: the paper's key claims rest on its own 818-participant studies and supplementary data, not on the cited prior work. No fitted parameter is relabeled as a prediction, no uniqueness theorem is imported from the authors' prior work, and no known empirical pattern is merely renamed as a taxonomy. Even if Experiment 3's manipulation is confounded—the lower-bias-perception condition also adds complexity and alternative outputs, and the higher-bias-perception condition removes axes—that is a causal-inference threat, not a circularity: the observed trust difference does not reduce to the definitions of the constructs. Overall, the paper is self-contained against external benchmarks and its reasoning is not circular.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claim does not depend on fitted parameters or invented entities; it depends on construct validity of the survey scales, on using SHAP as ground truth for feature attribution across visualization types, on the causal ordering assumed in the mediation model, and on the loan model being representative of biased ML systems.

assumptions (4)
  • domain assumption Self-reported Likert responses and multiple-choice answers validly measure trust, comprehension, and bias perception as defined in Section 2.2.
    The entire empirical chain rests on construct validity of the survey instruments in Figure 3; the authors ran pilots with over 200 participants but do not independently validate the scales against behavioral benchmarks.
  • domain assumption Feature attribution direction and magnitude as computed by SHAP form a valid ground truth for comparing all six visualizations, including those that do not present attribution (CP) or present rules (Anchors).
    Section 5.2.1 states: 'we ensured that all visualizations had the same feature importance ordering and contribution values for a given scenario (using the SHAP order as our default).' This imposes an attribution-centric ground truth on tools with different explanation semantics.
  • domain assumption The causal ordering in the RQ2 mediation model (comprehension -> bias perception -> trust) is correct and there are no unmeasured common causes.
    Section 6 fits a mediation model to observational survey data, which requires strong ignorability; the later experiments support the ordering, but the RQ2 estimates themselves rely on this assumption.
  • domain assumption The loan model and its sex and age biases are representative of 'biased ML models' for generalization.
    Section 5.2 uses a single LightGBM model on the Census Income dataset with sex as the primary discriminatory factor; Section 8 acknowledges that gender bias is widely documented and may prime participants to perceive bias.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Your Model Is Unfair, Are You Even Aware? Inverse Relationship Between Comprehension and Trust in Explainability Visualizations of Biased ML Models." pith.science (2026). https://pith.science/paper/P7GCONNH

@misc{pith2026250800140,
  author       = {Pith},
  title        = {Pith review of: Your Model Is Unfair, Are You Even Aware? Inverse Relationship Between Comprehension and Trust in Explainability Visualizations of Biased ML Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P7GCONNH}},
  note         = {Machine review of arXiv:2508.00140}
}
read the original abstract

Systems relying on ML have become ubiquitous, but so has biased behavior within them. Research shows that bias significantly affects stakeholders' trust in systems and how they use them. Further, stakeholders of different backgrounds view and trust the same systems differently. Thus, how ML models' behavior is explained plays a key role in comprehension and trust. We survey explainability visualizations, creating a taxonomy of design characteristics. We conduct user studies to evaluate five state-of-the-art visualization tools (LIME, SHAP, CP, Anchors, and ELI5) for model explainability, measuring how taxonomy characteristics affect comprehension, bias perception, and trust for non-expert ML users. Surprisingly, we find an inverse relationship between comprehension and trust: the better users understand the models, the less they trust them. We investigate the cause and find that this relationship is strongly mediated by bias perception: more comprehensible visualizations increase people's perception of bias, and increased bias perception reduces trust. We confirm this relationship is causal: Manipulating explainability visualizations to control comprehension, bias perception, and trust, we show that visualization design can significantly (p < 0.001) increase comprehension, increase perceived bias, and reduce trust. Conversely, reducing perceived model bias, either by improving model fairness or by adjusting visualization design, significantly increases trust even when comprehension remains high. Our work advances understanding of how comprehension affects trust and systematically investigates visualization's role in facilitating responsible ML applications.

Figures

Figures reproduced from arXiv: 2508.00140 by the authors.

Figure 1
Figure 1. The distributions and means of participants’ comprehension, bias perception, and trust in ML models across five explainability [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Our visualization design characteristic taxonomy, with 54 charac [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Overview of survey questions for our three metrics— [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The six state-of-the-art explainability visualizations in RQ2’s user study: (a) SHAP waterfall, (b) SHAP force, (c) ELI5, (d) LIME, (e) CP, and [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 6
Figure 6. Figure 6: A bar graph showing the difference in average trust between [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: A bubble chart showing the number of participants mentioning tax [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Three crowd-sourced controlled experiments testing for causality [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

110 extracted references · 30 canonical work pages

  1. [1]

    Ahn et al

    D. Ahn et al. Impact of model interpretability and outcome feedback on trust in AI. In CHI, 2024. doi: 10.1145/3613904.3642780 3

    arXiv 2024

  2. [2]

    Ahn and Y .-R

    Y . Ahn and Y .-R. Lin. Fairsight: Visual analytics for fairness in decision making. TVCG, p. 1–1, 2019. doi: 10.1109/tvcg.2019.2934262 2

    work page arXiv 2019

  3. [3]

    Does Explainable Artificial Intelligence Improve Human Decision-Making?

    Y . Alufaisan et al. Does explainable artificial intelligence improve human decision-making? arxiv.org/abs/2006.11194, 2020. 3

    work page Pith review arXiv 2006

  4. [4]

    Angwin et al

    J. Angwin et al. Machine bias. ProPublica, 2016. tinyurl.com/ 56pfaa6m. 1

    2016

  5. [5]

    Arya et al

    V . Arya et al. One explanation does not fit all: A toolkit and taxonomy of AI explainability techniques. arxiv.org/abs/1909.03012, 2019. 3

    arXiv 1909

  6. [6]

    J. P. Bagrow. Democratizing AI: Non-expert design of prediction tasks. PeerJ Computer Science, 6:e296, 2020. doi: 10.7717/peerj-cs.296 2

    work page doi:10.7717/peerj-cs.296 2020

  7. [7]

    Baniecki et al

    H. Baniecki et al. dalex: Responsible machine learning with interactive explainability and fairness in Python. JMLR, 22(214):1–7, 2021. 2, 5, 6

    2021

  8. [8]

    Bansal et al

    G. Bansal et al. Does the whole exceed its parts? The effect of AI explanations on complementary team performance. In CHI, 2021. doi: 10.1145/3411764.3445717 2, 3

    arXiv 2021

Show all 110 references
  1. [9]

    Bastani, C

    O. Bastani, C. Kim, and H. Bastani. Interpreting blackbox models via model extraction. arxiv.org/abs/1705.08504, 2019. 2

    2019 arXiv

  2. [10]

    Becker and R

    B. Becker and R. Kohavi. Census income. archive.ics.uci.edu/ dataset/2/adult, 1996. 5

    1996

  3. [11]

    Bhatt, M

    U. Bhatt, M. Andrus, A. Weller, and A. Xiang. Machine learning explain- ability for external stakeholders. arXiv:2007.05408, 2020. 1, 2

    2007 arXiv

  4. [12]

    P. Biecek. DALEX: Explainers for complex predictive models in R. JMLR, 19(84):1–5, 2018. 1

    2018

  5. [13]

    Biecek and T

    P. Biecek and T. Burzykowski. Explanatory Model Analysis. Chapman and Hall/CRC, New York, 2021. 2, 3, 5, 9

    2021

  6. [14]

    Blanco-Justicia and J

    A. Blanco-Justicia and J. Domingo-Ferrer. Machine learning explainabil- ity through comprehensible decision trees. In MLKE, 2019. 2

    2019

  7. [15]

    Buolamwini and T

    J. Buolamwini and T. Gebru. Gender Shades: Intersectional accuracy disparities in commercial gender classification. In FAccT, vol. 81, pp. 77–91. PMLR, 2018. 1

    2018

  8. [16]

    Burkart and M

    N. Burkart and M. F. Huber. A survey on the explainability of supervised machine learning. JAIR, 70:245–317, 2021. doi: 10.1613/jair.1.12228 2, 3

    2021 doi

  9. [17]

    Á. A. Cabrera et al. FairVis: Visual analytics for discovering intersec- tional bias in machine learning. In VAST, pp. 46–56, 2019. doi: 10. 1109/VAST47406.2019.8986948 1, 2

    2019

  10. [18]

    Colin, T

    J. Colin, T. Fel, R. Cadene, and T. Serre. What I cannot predict, I do not understand: A human-centered evaluation framework for explainability methods. arxiv.org/abs/2112.04417, 2023. 3

    2023 arXiv

  11. [19]

    Collaris and J

    D. Collaris and J. Wijk. Comparative evaluation of contribution-value plots for machine learning understanding.Journal of Visualization, 25:47– 57, 2021. doi: 10.1007/s12650-021-00776-w 3, 4

    2021 doi

  12. [20]

    J. W. Crandall et al. Cooperating with machines.Nature Communications, 9(1), 2018. doi: 10.1038/s41467-017-02597-8 2

    2018 doi

  13. [21]

    R. J. Crouser et al. Building and eroding: Exogenous and endogenous factors that influence subjective trust in visualization. In IEEE VIS, pp. 306–310, Oct. 2024. doi: 10.1109/VIS55277.2024.00069 3

    2024

  14. [22]

    W. K. Diprose et al. Physician understanding, explainability, and trust in a hypothetical machine learning risk calculator. JAMIA, 27(4):592–600,

  15. [23]

    Ehsan, Q

    U. Ehsan, Q. V . Liao, M. Muller, M. O. Riedl, and J. D. Weisz. Expanding explainability: Towards social transparency in ai systems. In CHI, 2021. doi: 10.1145/3411764.3445188 3

    2021

  16. [24]

    Elhamdadi, A

    H. Elhamdadi, A. Stefkovics, J. Beyer, E. Moerth, H. Pfister, C. X. Bearfield, and C. Nobre. Vistrust: A multidimensional framework and empirical study of trust in data visualizations. TVCG, 30(1):348–358, Jan. 2024. doi: 10.1109/TVCG.2023.3326579 3

    2024

  17. [25]

    Regulation (EU) 2016/679 of the European Parliament and of the Council

    European Parliament and Council of the European Union. Regulation (EU) 2016/679 of the European Parliament and of the Council. data. europa.eu/eli/reg/2016/679/oj, 2016. 2

    2016

  18. [26]

    Big data: A reporton algorithmic systems, opportunity, and civil rights.tinyurl.com/u4hjp587, 2016

    Executive Office of the President. Big data: A reporton algorithmic systems, opportunity, and civil rights.tinyurl.com/u4hjp587, 2016. 1

    2016

  19. [27]

    S. A. Friedler, C. Scheidegger, and S. Venkatasubramanian. The (im)possibility of fairness: Different value systems require different mechanisms for fair decision making. CACM, 64(4):136–143, Mar. 2021. doi: 10.1145/3433949 2

    2021 doi

  20. [28]

    A. Gaba, Z. Kaufman, J. Cheung, M. Shvakel, K. W. Hall, Y . Brun, and C. X. Bearfield. My model is unfair, do people even care? Visual design affects trust and perceived bias in machine learning. TVCG, 30(1):327–337, 2024. doi: 10.1109/TVCG.2023.3327192 1, 2, 3, 4, 6, 7, 9

    2024

  21. [29]

    Galhotra, Y

    S. Galhotra, Y . Brun, and A. Meliou. Fairness testing: Testing software for discrimination. In ESEC/FSE, pp. 498–510, 2017. doi: 10 .1145/3106237.3106277 2

    2017

  22. [30]

    E. D. Gennatas et al. Expert-augmented machine learning. PNAS, 117(9):4571–4577, 2020. doi: 10.1073/pnas.1906831117 2

    2020 doi

  23. [32]

    Goldstein, A

    A. Goldstein, A. Kapelner, J. Bleich, and E. Pitkin. Peeking inside the black box: Visualizing statistical learning with plots of individual con- ditional expectation. JCGS, 24(1):44–65, 2015. doi: 10.1080/10618600 .2014.907095 2

    2015

  24. [33]

    Guidotti

    R. Guidotti. Counterfactual explanations and how to find them: Literature review and benchmarking. Data Mining and Knowledge Discovery, pp. 1–55, 2022. doi: 10.1007/s10618-022-00831-6 2

    2022 doi

  25. [34]

    Hailesilassie

    T. Hailesilassie. Rule extraction algorithm for deep neural networks: A review. arxiv.org/abs/1610.05267, 2016. 2

    2016 arXiv

  26. [35]

    Hall and D

    P. Hall and D. Atherton. Awesome machine learning interpretability. tinyurl.com/2zaj8amsÆŠ, 2024. 3

    2024

  27. [36]

    Hartwig and C

    K. Hartwig and C. Reuter. Nudging users towards better security decisions in password creation using whitebox-based multidimensional visualisations. Behaviour & Information Technology, 41(7):1357–1380,

  28. [37]

    G. He, L. Kuiper, and U. Gadiraju. Knowing about knowing: An illusion of human competence can hinder appropriate reliance on ai systems. In CHI, 2023. doi: 10.1145/3544548.3581025 3

    2023

  29. [38]

    Heimerl, K

    A. Heimerl, K. Weitz, T. Baur, and E. André. Unraveling ML models of emotion with NOV A: Multi-level explainable AI for non-experts. TAFFC, 13(3):1155–1167, 2022. doi: 10.1109/TAFFC.2020.3043603 1, 2

    2022

  30. [39]

    J. Q. Ho, A. Hartanto, A. Koh, and N. M. Majeed. Gender biases within artificial intelligence and ChatGPT: Evidence, sources of biases and solutions. Computers in Human Behavior: Artificial Humans, 4:100145,

  31. [40]

    A. Hoag, J. E. Kostas, B. C. da Silva, P. S. Thomas, and Y . Brun. Seldonian toolkit: Building software with safe and fair machine learning. In ICSE Demo, 2023. doi: 10.1109/ICSE-Companion58688.2023.00035 1

    2023

  32. [41]

    M. N. Hoque and K. Mueller. Outcome-explorer: A causality guided interactive visual interface for interpretable algorithmic decision making. TVCG, 28(12):4728–4740, 2022. doi: 10.1109/tvcg.2021.3102051 3, 9

    2022

  33. [42]

    S. R. Islam, W. Eberle, S. K. Ghafoor, and M. Ahmed. Explainable artifi- cial intelligence approaches: A survey. arxiv.org/abs/2101.09429,

    arXiv

  34. [43]

    Jeong, S

    D. Jeong, S. Aggarwal, J. Robinson, N. Kumar, A. Spearot, and D. S. Park. Exhaustive or exhausting? evidence on respondent fatigue in long surveys. Journal of Development Economics, 161:102992, 2023. doi: 10.1016/j.jdeveco.2022.102992 9

    2023

  35. [44]

    Johnson and Y

    B. Johnson and Y . Brun. Fairkit-learn: A fairness evaluation and comparison toolkit. In ICSE Demo, 2022. doi: 10.1145/3510454.3516830 2

    2022

  36. [45]

    Johnson, Y

    B. Johnson, Y . Brun, and A. Meliou. Causal testing: Understanding defects’ root causes. In ICSE, 2020. doi: 10.1145/3377811.3380377 2

    2020

  37. [46]

    Johnson et al

    B. Johnson et al. Fairkit, fairkit, on the wall, who’s the fairest of them all? Supporting data scientists in training fair models. EURO Journal on Decision Processes, 11, 2023. doi: 10.1016/j.ejdp.2023.100031 1, 2

    2023

  38. [47]

    Kaufman, M

    Z. Kaufman, M. Endres, C. Xiong Bearfield, and Y . Brun. Sup- plementary materials. https://osf.io/c87xm/?view_only= 31dfc1f2a7624f5cb20b0f07d3730df3. 2, 3, 5

  39. [48]

    Kaur et al

    H. Kaur et al. Interpreting interpretability: Understanding data scientists’ use of interpretability tools for machine learning. In CHI, p. 1–14, 2020. doi: 10.1145/3313831.3376219 3

    2020

  40. [49]

    Ke et al

    G. Ke et al. LightGBM: A highly efficient gradient boosting decision tree. In NeurIPS, 2017. 5

    2017

  41. [50]

    S. S. Y . Kim, N. Meister, V . V . Ramaswamy, R. Fong, and O. Russakovsky. HIVE: Evaluating the human interpretability of visual explanations. In ECCV, p. 280–298, 2022. doi: 10.1007/978-3-031-19775-8_17 3

    2022 doi

  42. [51]

    Klaise, A

    J. Klaise, A. V . Looveren, G. Vacanti, and A. Coca. Alibi explain: Al- gorithms for explaining machine learning models. JMLR, 22(1), 2021. 2

    2021

  43. [52]

    Komorowski, L

    M. Komorowski, L. A. Celi, O. Badawi, A. C. Gordon, and A. A. Faisal. The artificial intelligence clinician learns optimal treatment strategies for sepsis in intensive care. Nature Medicine, 24(11):1716–1720, 2018. 1, 2

    2018

  44. [53]

    Korobov and K

    M. Korobov and K. Lopuhin. ELI5. github.com/eli5-org/eli5/ tree/master, 2016–2017. 1, 3, 5, 6

    2016

  45. [54]

    Krishna et al

    S. Krishna et al. The disagreement problem in explainable machine learning: A practitioner’s perspective. arxiv.org/abs/2202.01602,

    arXiv

  46. [55]

    M. K. Lee and K. Rich. Who is included in human perceptions of AI?: Trust and perceived fairness around healthcare AI and cultural mistrust. In CHI, 2021. doi: 10.1145/3411764.3445570 3, 9

    2021

  47. [56]

    Liang, J

    A. Liang, J. Lu, and X. Mu. Algorithm design: A fairness-accuracy frontier. arxiv.org/abs/2112.09975, 2023. 2

    2023 arXiv

  48. [57]

    Z. C. Lipton. The mythos of model interpretability. arxiv.org/abs/1606.03490, 2017. 2

    2017 arXiv

  49. [58]

    M. Liu, J. Shi, K. Cao, J. Zhu, and S. Liu. Analyzing the training processes of deep generative models. TVCG, 24(1):77–87, 2018. doi: 10.1109/TVCG.2017.2744938 2

    2018

  50. [59]

    S. M. Lundberg et al. Explainable machine-learning predictions for the prevention of hypoxaemia during surgery. Nature Biomedical Engineering, 2(10):749, 2018. 1, 3, 5, 6

    2018

  51. [60]

    S. M. Lundberg and S.-I. Lee. A unified approach to interpreting model predictions. In NeurIPS, vol. 30, 2017. 1, 2, 3, 5, 6

    2017

  52. [61]

    Ma et al

    S. Ma et al. Who should I trust: AI or myself? leveraging human and AI correctness likelihood to promote appropriate trust in AI-assisted decision-making. arxiv.org/abs/2301.05809, 2023. 3

    2023 arXiv

  53. [62]

    Maksymiuk, A

    S. Maksymiuk, A. Gosiewska, and P. Biecek. Landscape of R packages for eXplainable AI. arxiv.org/abs/2009.13248, 2021. 3

    2009 arXiv

  54. [63]

    Mertes, T

    S. Mertes, T. Huber, K. Weitz, A. Heimerl, and E. André. Ganter- factual—counterfactual explanations for medical non-experts using generative adversarial learning. Frontiers in AI, 5:825565, 2022. 1, 2

    2022

  55. [64]

    Lightgbm

    Microsoft. Lightgbm. github.com/microsoft/LightGBM, 2017. 5

    2017

  56. [65]

    Y . Ming, H. Qu, and E. Bertini. Rulematrix: Visualizing and under- standing classifiers with rules. TVCG, 25(1):342–352, 2019. doi: 10 .1109/TVCG.2018.2864812 2, 3

    2019

  57. [66]

    Nakao et al

    Y . Nakao et al. Toward involving end-users in interactive human-in-the- loop AI fairness. ACM TiiS, 12(3), July 2022. doi: 10.1145/3514258 4

    2022 doi

  58. [67]

    Nakao and Y

    Y . Nakao and Y . Sugano. Use of machine learning by non-expert DHH people: Technological understanding and sound perception. In NordiCHI, 2020. doi: 10.1145/3419249.3420157 2

    2020

  59. [68]

    Nauta et al

    M. Nauta et al. From anecdotal evidence to quantitative evaluation methods: A systematic review on evaluating explainable AI. ACM Computing Surveys, 55(13s):1–42, 2023. doi: 10.1145/3583558 3

    2023 doi

  60. [69]

    R. C. Nickerson, U. Varshney, and J. Muntermann. A method for taxonomy development and its application in information systems. European Journal of Information Systems, 22:336–359, 2013. 3

    2013

  61. [70]

    Özalp, K

    E. Özalp, K. Hartwig, and C. Reuter. Trends in explainable artificial intel- ligence for non-experts.KI-Kritik/AI Critique Volume 4, p. 223, 2023. 1, 2

    2023

  62. [71]

    Palan and C

    S. Palan and C. Schitter. Prolific.ac — A subject pool for online experiments. JBEF, 17:22–27, 2018. 4, 6

    2018

  63. [72]

    RStudio desktop

    Posit. RStudio desktop. posit.co/download/rstudio-desktop/,

  64. [74]

    How intelligent machines are reshaping investing

    Q.ai. How intelligent machines are reshaping investing. Forbes, 2022. 1, 2

    2022

  65. [75]

    Qualtrics

    I. Qualtrics. Qualtrics. Provo, UT, USA, 2013. 4

    2013

  66. [76]

    Rechkemmer and M

    A. Rechkemmer and M. Yin. When confidence meets accuracy: Exploring the effects of multiple performance indicators on trust in machine learning models. In CHI, 2022. doi: 10.1145/3491102.3501967 3

    2022

  67. [77]

    Why should I trust you?

    M. T. Ribeiro, S. Singh, and C. Guestrin. “Why should I trust you?”: Explaining the predictions of any classifier. In KDD, pp. 1135–1144,

  68. [78]

    M. T. Ribeiro, S. Singh, and C. Guestrin. Anchors: High-precision model-agnostic explanations. In AAAI, 2018. 1, 3, 5, 6

    2018

  69. [79]

    Rong et al

    Y . Rong et al. Towards human-centered explainable AI: A survey of user studies for model explanations. TPAMI, 46(4):2104–2122, 2024. doi: 10.1109/TPAMI.2023.3331846 3

    2024

  70. [80]

    P. K. Roy, S. S. Chowdhary, and R. Bhatia. A machine learning approach for automation of resume recommendation system. Procedia Computer Science, 167:2318–2327, 2020. 1, 2

    2020

  71. [81]

    Salehin et al

    I. Salehin et al. AutoML: A systematic review on automated machine learning with neural architecture search. Journal of Information and Intelligence, 2(1):52–81, 2024. doi: doi.org/10.1016/j.jiixd.2023.10.002 2

    2024 doi

  72. [82]

    Samek, T

    W. Samek, T. Wiegand, and K.-R. Müller. Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models. arxiv.org/abs/1708.08296, 2017. 2

    2017 arXiv

  73. [83]

    Sheard and A

    N. Sheard and A. Schwartz. The movement to ban government use of face recognition. tinyurl.com/56tfjkjs, 2022. 1

    2022

  74. [84]

    Shen and T.-H

    H. Shen and T.-H. K. Huang. How useful are the machine-generated interpretations to general users? A human evaluation on guessing the incorrectly predicted labels. In HCOMP, 2020. 3

    2020

  75. [85]

    N. Singer. Amazon faces investor pressure over facial recognition. NYT,

  76. [86]

    Staniak and P

    M. Staniak and P. Biecek. Explanations of model predictions with live and breakDown packages. The R Journal, 2018. doi: 10.32614/RJ-2018-072 2

    2018 doi

  77. [87]

    Stepin, J

    I. Stepin, J. M. Alonso, A. Catala, and M. Pereira-Fariña. A survey of contrastive and counterfactual explanation generation methods for explainable artificial intelligence. IEEE Access, 9:11974–12001, 2021. doi: 10.1109/ACCESS.2021.3051315 2

    2021

  78. [88]

    P. S. Thomas, B. C. da Silva, A. G. Barto, S. Giguere, Y . Brun, and E. Brunskill. Preventing undesirable behavior of intelligent machines. Science, 366(6468):999–1004, 2019. doi: 10.1126/science.aag3311 1, 2, 8

    2019 doi

  79. [89]

    Tolmeijer, M

    S. Tolmeijer, M. Christen, S. Kandul, M. Kneer, and A. Bernstein. Capable but amoral? Comparing AI and human expert collaboration in ethical decision making. In CHI, 2022. doi: 10.1145/3491102.3517732 3

    2022

  80. [90]

    van Berkel, J

    N. van Berkel, J. Goncalves, D. Russo, S. Hosio, and M. B. Skov. Effect of information presentation on fairness perceptions of machine learning predictors. In CHI, 2021. doi: 10.1145/3411764.3445365 2, 3

    2021

  81. [91]

    Vanderford

    R. Vanderford. New York’s landmark AI bias law prompts uncertainty. WSJ, Sept. 21, 2022. tinyurl.com/wv4ueczf. 1

    2022

  82. [92]

    Veale and R

    M. Veale and R. Binns. Fairer machine learning in the real world: Mitigating discrimination without collecting sensitive data. Big Data & Society, 4(2), 2017. 2

    2017

  83. [93]

    D. Wang, W. Zhang, and B. Y . Lim. Show or suppress? Man- aging input uncertainty in machine learning model explanations. arxiv.org/abs/2101.09498, 2021. 2

    2021 arXiv

  84. [94]

    Q. Wang, K. Huang, P. Chandak, M. Zitnik, and N. Gehlenborg. Extend- ing the nested model for user-centric xai: A design study on gnn-based drug repurposing. TVCG, PP, 10 2022. doi: 10.1109/TVCG.2022.3209435 3

    2022

  85. [95]

    Q. Wang, Z. Xu, Z. Chen, Y . Wang, S. Liu, and H. Qu. Visual analysis of discrimination in machine learning. TVCG, 27(2):1470–1480, 2021. doi: 10.1109/TVCG.2020.3030471 2, 4

    2021

  86. [96]

    Wang et al

    R. Wang et al. Factors influencing perceived fairness in algorithmic decision-making: Algorithm outcomes, development procedures, and in- dividual differences. In CHI, 2020. doi: 10.1145/3313831.3376813 2, 3, 6

    2020

  87. [97]

    Wexler et al

    J. Wexler et al. The what-if tool: Interactive probing of machine learning models. TVCG, p. 1–1, 2019. doi: 10.1109/tvcg.2019.2934619 2

    2019

  88. [98]

    Wielopolski, O

    P. Wielopolski, O. Furman, J. Stefanowski, and M. Zieba. Unifying per- spectives: Plausible counterfactual explanations on global, group-wise, and local levels. https://arxiv.org/abs/2405.17642, 2024. 2

    2024 arXiv

  89. [99]

    T. Xie, Y . Ma, J. Kang, H. Tong, and R. Maciejewski. FairRankVis: A visual analytics framework for exploring algorithmic fairness in graph mining models. TVCG, 28(1), 2022. doi: 10.1109/TVCG.2021.3114850 2

    2022

  90. [100]

    Xiong et al

    C. Xiong et al. Illusion of causality in visualized data. TVCG, 26(1):853–862, 2020. doi: 10.1109/TVCG.2019.2934399 2, 3

    2020

  91. [101]

    Xiong et al

    C. Xiong et al. Reasoning affordances with tables and bar charts. TVCG, pp. 1–13, 2022. doi: 10.1109/TVCG.2022.3232959 2, 3

    2022

  92. [102]

    Xiong, L

    C. Xiong, L. Padilla, K. Grayson, and S. Franconeri. Examining the components of trust in map-based visualizations. In TrustVis, 2019. 6

    2019

  93. [103]

    A. Yala, C. Lehman, T. Schuster, and T. P. andRegina Barzilay. A deep learning mammography-based model for improved breast cancer risk prediction. Radiology, 2019. doi: 10.1148/radiol.2019182716 1

    2019 doi

  94. [104]

    Yang et al

    F. Yang et al. Swaying the public? Impacts of election forecast visualizations on emotion, trust, and intention in the 2022 U.S. midterms. TVCG, 30(01):23–33, 2024. doi: 10.1109/TVCG.2023.3327356 3

    2022

  95. [105]

    F. Yang, Z. Huang, J. Scholtz, and D. L. Arendt. How do visual explanations foster end users’ appropriate trust in machine learning? In IUI, pp. 189–201, 2020. doi: 10.1145/3377325.3377480 3

    2020

  96. [106]

    Q. Yang, J. Suh, N.-C. Chen, and G. Ramos. Grounding interactive machine learning tool design in how non-experts actually build models. In DIS, pp. 573–584, 2018. doi: 10.1145/3196709.3196729 2

    2018

  97. [107]

    M. Yin, J. Wortman Vaughan, and H. Wallach. Understanding the effect of accuracy on trust in machine learning models. In CHI, pp. 1–12, 2019. doi: 10.1145/3290605.3300509 3

    2019

  98. [108]

    Zhang, Q

    Y . Zhang, Q. V . Liao, and R. K. E. Bellamy. Effect of confidence and explanation on accuracy and trust calibration in ai-assisted decision making. In FAT*, p. 295–305, 2020. doi: 10.1145/3351095.3372852 3

    2020

  99. [2016]

    doi: 10.1145/2939672.2939778 1, 2, 3, 5, 6

  100. [2020]

    doi: 10.1093/jamia/ocz229 2

    doi

  101. [2022]

    doi: 10.1080/0144929X.2021.1876167 2

    2021

  102. [2025]

    doi: 10.1016/j.chbah.2025.100145 9

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.