Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Can Global XAI Methods Reveal Injected Behaviours in LLMs? SHAP vs Rule Extraction vs RuleSHAP

T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Global XAI on numerical scores of an LLM's self-reported beliefs can recover injected bias-triggering heuristics as symbolic rules, and RuleSHAP finds them more often than RuleFit or SHAP.

desk verdict RuleSHAP is a plausible incremental improvement over RuleFit for rule extraction, but the injected benchmark guarantees the very association it claims to discover; treat the narrow algorithm comparison as the contribution. read the letter →

arxiv 2505.11189 v3 pith:SW7JFUHE submitted 2025-05-16 cs.AI cs.LG

Francesco Sovrano This is my paper · ORCID
classification cs.AIcs.LG
keywords explainableAIruleextractionSHAPLLMbeliefsmisinformationglobalexplanationslargelanguagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the belief-driven shortcuts behind some LLM misbehavior can be pulled out as readable if-then rules, even though the models only take text in and out. It does this by first turning topics into numbers: the LLM rates each topic on scales like commonality, positivity, and controversy, and the explanations it generates are scored for length, readability, sentiment, and subjectivity. On these numeric abstractions, ordinary global explanation tools can run; the paper shows that SHAP flags the right features but gives no rules, RuleFit gives rules but misses many, and a new hybrid called RuleSHAP recovers injected univariate, conjunctive, and non-convex trigger rules more reliably. Across five GPT and Llama models, RuleSHAP improves top-1 rule recovery over RuleFit by about 82 percent on average, with p-values below 0.003. If this holds, auditors could surface an LLM's behavioural triggers without touching its weights.

What carries the argument

The load-bearing machinery is a two-part abstraction-plus-rule-induction pipeline. First, each topic is mapped to an 11-dimensional vector $u$ of the LLM's self-reported belief scores (1–5) on properties like commonality, positivity, controversy, conceptual density, interdisciplinarity, and temporal or geographic variability, while each generated explanation is mapped to a 7-dimensional vector $v$ of length, Gunning fog readability, sentiment, subjectivity, and three LLM-as-judge bias ratings. Second, RuleSHAP couples global SHAP aggregates with rule induction: it computes per-feature Shapley values over all points, aggregates them as mean absolute value plus standard deviation, normalizes to $(0,1]$, uses these as feature weights in XGBoost (with `colsample_bylevel` set so roughly one feature is sampled per split), and then runs a LASSO whose penalty for each candidate rule is scaled inversely by the average SHAP weight of the features in that rule. The design makes global SHAP steer both the tree construction and the sparse rule selection, while dropping the support-based weighting that RuleFit uses for rule importance.

What would settle it

Run RuleSHAP on a model whose injected trigger uses a feature that is deliberately absent from the elicited belief vectors—for example, make response length depend on the number of words in the topic phrase while all 11 rated properties are held approximately constant. If RuleSHAP still surfaces the injected rule, the method recovers triggers independent of the abstraction; if it does not, the recovered rules are confined to the pre-chosen belief dimensions. A second check: hold out part of the topic set, fit the abstraction correlation certificates only on the training half, and verify whether the extracted top rules still rank injected triggers on the held-out half at the same MRR.

Watch

Extended reading notes

Core claim

The paper's central claim is that a statistically validated mapping from text to numbers—the LLM's own 1-to-5 belief ratings of topic properties such as commonality, positivity, controversy, and interdisciplinarity, paired with seven numeric output metrics (length, readability, sentiment, subjectivity, and three LLM-as-judge bias estimates)—makes LLM behaviour accessible to global XAI, and that the belief-driven heuristics behind misinformation-related biases can be recovered as explicit rules. To get ground truth, the authors inject 14 bias rules of three complexity levels (univariate, conjunctive, non-convex) through system-instruction wrappers, and measure how often each XAI method ranks the injected rule at the top (MRR@1, MRR@3, MRR@10). They report that RuleFit under-detects non-univariate triggers, that global SHAP ranks them better but cannot express them symbolically, and that RuleSHAP—global SHAP aggregates feeding both XGBoost split sampling and a SHAP-aware LASSO—detects more of them with more compact rule sets. Correlation certificates (distance correlation for the non-linear injections) support that the numeric abstractions capture real input-output regularities, and an uninjected case study on 30,000-plus SDG-related topics finds systematic rules such as common topics being simplified and positive topics being answered more positively.

Load-bearing premise

The load-bearing assumption is that an LLM's self-reported 1-to-5 belief scores for a topic are stable, causally upstream features that determine the bias-related properties of its outputs; if those ratings do not mirror the internal representations that actually shape generation, the extracted rules describe correlations between the model's own questionnaire answers and output proxies, not the mechanisms behind the behaviour.

Editorial extensions

If this is right

  • An LLM auditor can run the whole pipeline without access to weights or hidden states: only topic prompts, generated texts, and the model's own ratings are needed.
  • Because RuleSHAP cuts rule overproduction by about 10.9 percent on average while raising faithfulness, rule-set size and detection quality do not trade off in the tested settings.
  • Conjunctive triggers (e.g., low commonality and moderate positivity) are the clearest win: RuleSHAP's MRR@1 on conjunctive rules is roughly 0.72 versus 0.32 for RuleFit, so multi-feature heuristics become findable rather than missed.
  • Non-convex triggers such as the interdisciplinarity-in-{1,3,5} modulo condition remain the hardest case for every tested method, implying that rule extraction still has a complexity ceiling even for simple injected heuristics.
  • On uninjected SDG-topic data, the same pipeline surfaces recurring belief-driven patterns—simplified language for common topics, sentiment echoing a topic's perceived valence, longer texts for technical topics—so the method is not only validated on synthetic injections.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • I infer the method's portability hinges on the stability of the belief elicitation: if rewording the rating prompts changes the extracted rules, then the rules describe the model's answers to a particular questionnaire rather than a stable behavioural trigger. This is testable by re-running the pipeline with two differently worded belief-elicitation templates and comparing rule sets.
  • The reliance on in-sample correlation certificates means the reported effect sizes are partly built into the injected data, since the behavioural wrapper reads the same $u$ scores used for explanation; an out-of-sample or shuffled-$u$ control would be needed to separate genuine trigger recovery from guaranteed correlation.
  • The SHAP-weighting trick generalizes in principle to any tree-based surrogate and any sparse linear selector, so RuleSHAP's design could be lifted into other rule-extraction toolchains, not just XGBoost-plus-LASSO.
  • The non-convex failures suggest that the next bottleneck is representational: parity- or set-membership conditions may need engineered interaction features or modulo-aware abstractions before global rules can be recovered.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a pipeline for global rule extraction from LLMs. It maps topics to an 11-dimensional belief vector u (elicited from the LLM itself) and outputs to 7 proxy/judge scores v, then applies global XAI methods on (u, v). To obtain ground truth, the authors inject three families of bias rules (univariate, conjunctive, non-convex) via system instructions that are triggered by the same u vectors. They introduce RuleSHAP, which couples global SHAP aggregations with XGBoost feature weighting and SHAP-aware LASSO. Across five LLMs and SDG-related topics, RuleSHAP is reported to outperform RuleFit, SHAP, and decision-tree baselines on mean reciprocal rank (MRR@1 average 0.568 vs. 0.31 for RuleFit) while producing more compact rule sets. The paper also reports correlation certificates and a real-data case study on LLM explanations of SDG topics.

Significance. If the central claims are taken at face value, RuleSHAP would be a useful model-agnostic audit tool for LLM bias detection, and the paper would contribute a reproducible benchmark for global XAI methods in the text domain. The manuscript has clear strengths: a public replication package, an a priori power analysis, evaluation across five LLMs, a three-tier complexity design, and a careful ablation isolating the contribution of the SHAP-guided XGBoost step. The narrow algorithmic comparison (RuleSHAP vs. RuleFit on the injected benchmark) is supported by Table 2 and the reported Wilcoxon tests. However, the paper's broader title claim -- that the method reveals 'injected behaviours in LLMs' as belief-driven heuristics -- is not supported by the experimental design, because the injected wrapper reads exactly the same u features that are given to the XAI methods. The significance of the work is therefore contingent on substantially reframing or strengthening the claims.

major comments (4)
  1. [§3, §5, Appendix D] The pipeline's main assumption (§3) is that the self-reported belief vector u is a stable, causally upstream representation of the mechanism shaping output abstractions v. The injected ground truth, however, is a Python wrapper that reads the exact same u vectors produced by ϕx and prepends system instructions based on threshold conditions on those u values (Appendix D). Consequently, the u-to-v association is guaranteed by construction, and any rule-extraction method that sees u can recover the wrapper's threshold rules. The correlation certificates in §6 are computed in-sample on the same u and v, so they provide no independent evidence that u encodes internal LLM beliefs. To support the central claim, the authors must either inject rules based on variables not present in u (e.g., topic metadata or labels independent of the LLM's self-reports) or provide independent evidence (e.g., activation-level probes or controlled manipulation of internal representations) that u is causally upstream of v. Without this, the paper demonstrates wrapper-rule recovery, not belief-driven behavior discovery.
  2. [§6, Table 2] No held-out split is described anywhere in the evaluation. The MRR metrics appear to be computed on the same topic set that was used to fit the rule-extraction surrogates (XGBoost and LASSO). Given that the candidate rule sets contain thousands of rules and the injected rules are simple threshold functions over an 11-dimensional u, in-sample evaluation can substantially overestimate MRR through overfitting. The authors should report out-of-sample MRR via topic-level cross-validation or repeated resampling, and should provide confidence intervals over multiple seeds or repeated trials. Without such an evaluation, the claimed +82% MRR@1 advantage over RuleFit, while plausible, is not yet robustly established.
  3. [§6, Appendix I, Appendix K] The 'correlation certificates' are presented as validating the abstractions, but for the injected data they are tautological: the wrapper reads u, so high distance correlations between u and v are structurally guaranteed. For the real-data case study, the certificates are unthresholded: with N≈600–10,000 per LLM, distance correlations as low as dCorr=0.058 (§I.3) achieve p<0.001. Appendix K itself flags a rule with correlation near 0.05 as a possible hallucination. The paper should define a minimum effect size that qualifies as a 'correlation certificate' and apply it consistently to accept or reject rules. As written, the phrase 'statistically validated abstractions' overstates what the significance tests establish.
  4. [§3, Appendix B.2, §7 Limitations] Three of the seven output features are LLM-as-a-judge scores for framing, information overload, and oversimplification. The authors acknowledge in the Limitations that these 'may be judge-biased and introduce invalid circularity.' This circularity is material to the main comparison: the injected wrapper changes length, subjectivity, and readability, and the judge is explicitly asked to rate those biases, so rules over the judge-based targets can be induced by the wrapper even if the LLM's internal beliefs play no causal role. The authors argue that the judge scores correlate with independent proxies, but those proxies are also changed by the wrapper, so this correlation does not break the circularity. The analysis should either be repeated with only the four LLM-independent output features, or the results for judge-based targets should be reported separately and clearly qualified.
minor comments (4)
  1. [Abstract vs. Introduction] The abstract reports an average MRR@1 improvement of +82% over RuleFit, while the introduction (p. 2) reports +94% (MRR@1) on average. Please reconcile these numbers with Table 2 and Appendix J and specify the exact aggregation used for each figure.
  2. [Tables 2 and 6] The text rendering of Tables 2 and 6 contains formatting artifacts (e.g., '29230.14' and '9430.35' on row breaks). Please ensure the final PDF renders rule counts and MRR values as separate, unambiguous columns.
  3. [§4 Step 1] The description of SHAP background values and the nearest-neighbour matching procedure (finding j with ∥buk−uj∥2 minimal) is non-standard and should be clarified: in particular, how the minimum feature values from T are used as background, and how the matching procedure interacts with SHAP's perturbation distribution.
  4. [§6, Appendix I] The term 'correlation certificate' is used informally. I suggest giving a formal definition of what constitutes a valid certificate (test, effect-size threshold, and whether it is used for rule filtering or only for post-hoc validation) and stating how certificates are applied to accept or red-flag rules.

Circularity Check

2 steps flagged · score 5.0 of 10

Injected ground truth is a wrapper that reads the same u vectors used as XAI features, so the correlation certificates certify the wrapper; the broader belief-driven claim is partly circular, though the RuleSHAP-vs-baseline benchmark remains independent.

  1. self definitional [Section 3 'Input abstractions'; Section 5 'Injecting bias via system instructions'; Appendix D]
    "Because ui encodes the LLM’s belief about τi, it follows that ϕx(·) is, by definition, the LLM itself. ... It consists of a Python wrapper around f(xi) sending bj if the input abstraction scores produced by ϕx(τi) satisfy the preconditions required by bj. ... Since ground-truth biases are injected based on LLM abstractions, their existence is guaranteed, making XAI evaluations fully independent of hallucinations and prompt template choices."

    The 11 input features u are the LLM's own 1–5 self-reported belief scores. The injected 'ground-truth' bias is not an emergent LLM heuristic; it is a Python wrapper that tests conditions on exactly those u values and then appends a system instruction to change length, subjectivity, or readability. Hence the u-to-v regularity that SHAP, RuleFit, and RuleSHAP recover is inserted by the experimenter, not discovered as evidence that self-reported beliefs causally drive generation. The correlation certificates computed on these data are mathematically guaranteed by the wrapper, so they cannot validate the abstraction hypothesis; they only certify that the wrapper was applied.

  2. other [Section 6 'Correlation Certificates' and Section 7 'Limitations']
    "Correlation certificates show that the extracted rules capture meaningful links between input abstractions and output metrics. ... Second, to assess bias in AI-generated explanations, we use three LLM-as-a-judge abstractions, which may be judge-biased and introduce invalid circularity; however, their scores correlate strongly with LLM-independent proxies, vouching for their validity."

    The certificates are in-sample on the same u and v used to extract rules, and for injected data they are tautological because the wrapper enforces the association. The three LLM-as-a-judge output features are part of v and are generated by an LLM rating the very explanations that the wrapper has altered; if the judge inherits the manipulation, rules targeting these outputs partly encode the judge artifact. The paper's rebuttal (correlation with independent proxies) is itself in-sample and does not break the circularity, though the author explicitly flags the risk.

full rationale

The algorithm-level comparison (RuleSHAP vs RuleFit vs SHAP on injected rules) is a legitimate controlled benchmark: the XAI methods are not given the injected rules, and the MRR differences are real. If the paper only claimed 'RuleSHAP recovers wrapper rules better than RuleFit', there would be no circularity. The circularity enters when the same setup is used to support the broader claim that self-reported LLM beliefs causally drive bias-related behaviour and that correlation certificates validate the abstractions. For injected data the wrapper reads the very u vectors used as XAI features, so the u-v association and the certificates are guaranteed by construction; for the real-data case study the certificates are in-sample and, as the paper concedes, some correlations are near 0.05 and may be hallucinations. The LLM-as-a-judge output features are explicitly flagged by the author as potentially introducing invalid circularity; the rebuttal relies on in-sample correlations. None of these issues undermines the narrow comparative MRR result, so the score is moderate (5/10), not extreme.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The method's contribution is bounded by hand-chosen abstraction features, post hoc SHAP aggregation selection, and unverified hyperparameters. The most important unpaid premises are that self-reported belief scores are causally relevant features and that judge-based bias scores are valid; the latter is explicitly flagged as potentially circular by the author.

free parameters (3)
  • SHAP aggregation statistic = mean(|rho|) + std(|rho|)
    Selected in Section 6 because this upper-bound aggregation yielded the highest MRR empirically; this is a post hoc choice on the evaluation data.
  • RuleSHAP hyperparameters = colsample_bylevel = 1/M, tree_method = exact, alpha not reported
    Hand-set in Section 4; the LASSO sparsity alpha and other XGBoost settings are not given, so their influence on the reported MRR is uncontrolled.
  • Abstraction feature sets = 11 input properties, 7 output features
    The choice of which topic properties and output metrics to score determines which rules can be discovered; these are hand-designed operationalizations, not derived from theory.
assumptions (5)
  • domain assumption LLMs encode global beliefs and act as bags of heuristics
    Invoked in Section 1 via refs [9,10] to justify using LLM self-reported scores as explanatory features for behavior.
  • domain assumption Topics with identical u vectors yield similar output v vectors (abstraction validity)
    Stated as the main assumption in Section 3; sample size and correlation certificates are used to support it, but certificates for injected data are guaranteed by construction.
  • domain assumption LLM self-reported Likert scores are faithful ordered measures of topic properties
    The phi_x mapping asks the same LLM to rate topics on 1-5 scales, and the paper treats these ratings as the LLM's beliefs.
  • domain assumption Deterministic decoding (temperature 0, top-p 0) makes outputs reflect instructions and beliefs rather than randomness
    Used in Section 5 and Appendices A/B to justify stable correlations and certificates.
  • domain assumption LLM-as-a-judge bias scores are valid estimates of framing, overload, and oversimplification
    Defined in Section 3 and B.2; the author acknowledges in Limitations that this may introduce invalid circularity, with correlation to proxies as the only mitigation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Can Global XAI Methods Reveal Injected Behaviours in LLMs? SHAP vs Rule Extraction vs RuleSHAP." pith.science (2026). https://pith.science/paper/SW7JFUHE

@misc{pith2026250511189,
  author       = {Pith},
  title        = {Pith review of: Can Global XAI Methods Reveal Injected Behaviours in LLMs? SHAP vs Rule Extraction vs RuleSHAP},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SW7JFUHE}},
  note         = {Machine review of arXiv:2505.11189}
}
read the original abstract

Large language models (LLMs) can amplify misinformation, undermining societal goals such as the UN SDGs. We study three documented drivers of misinformation (valence framing, information overload, and oversimplification) often shaped by default beliefs. Building on evidence that LLMs encode such defaults (e.g., "joy is positive", "math is complex") and can act as "bags of heuristics", we ask whether belief-driven heuristics behind misinformation-related behaviour can be recovered from black-box LLM behaviour as explicit rules. A key obstacle is that global rule-extraction methods in explainable AI (XAI) are built for numerical input-output data, not text. We address this by eliciting global LLM beliefs and mapping them to numerical scores via statistically validated abstractions, enabling off-the-shelf global XAI to detect belief-driven heuristics. For ground truth, we inject nonlinear behavioural triggers of increasing complexity (univariate, conjunctive, non-convex) into GPT-family and Llama models via system instructions. We find that RuleFit often misses non-univariate triggers, while global SHAP better ranks conjunctive trigger features but yields no symbolic rules. To bridge this gap, we propose RuleSHAP, a rule-extraction algorithm that couples global SHAP aggregates with rule induction to better capture non-univariate triggers, improving MRR@1 over RuleFit by +82% on average. Our results suggest a practical pathway for surfacing behavioural triggers in LLMs.

Figures

Figures reproduced from arXiv: 2505.11189 by the authors.

Figure 1
Figure 1. Experiment pipeline: Green: ground truth injection; Yellow: data auto-generation. [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Neuron-Anchored Rule Extraction for Large Language Models via Contrastive Hierarchical Ablation

    cs.LG 2026-05 unverdicted novelty 7.0 of 10

    MechaRule localizes sparse agonist neurons via contrastive hierarchical ablation and adaptive group testing to ground rule extraction, recalling 97% of high-effect activations at 2.14% cost while enabling near-total e...

Reference graph

Works this paper leans on

57 extracted references · 36 canonical work pages · cited by 1 Pith paper

  1. [1]

    Sustainable development goals

    United Nations. Sustainable development goals. https://sdgs.un.org/goals, 2015. Ac- cessed: 2025-05-14

    work page 2015

  2. [2]

    The role of artificial intelligence in achieving the sustainable development goals.Nature communications, 11(1):1–10, 2020

    Ricardo Vinuesa, Hossein Azizpour, Iolanda Leite, Madeline Balaam, Virginia Dignum, Sami Domisch, Anna Felländer, Simone Daniela Langhans, Max Tegmark, and Francesco Fuso Nerini. The role of artificial intelligence in achieving the sustainable development goals.Nature communications, 11(1):1–10, 2020

    work page 2020

  3. [3]

    Schäfer, Afra Amini, Heidi Lam, Massimiliano Ciaramita, Ben Gaiarin, Michelle Chen Huebscher, Christian Buck, Niels Mede, Markus Leippold, and Nadine Strauß

    Jannis Bulian, Mike S. Schäfer, Afra Amini, Heidi Lam, Massimiliano Ciaramita, Ben Gaiarin, Michelle Chen Huebscher, Christian Buck, Niels Mede, Markus Leippold, and Nadine Strauß. Assessing large language models on climate information. InForty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net...

    work page 2024

  4. [4]

    Cognitive biases and artificial intelligence.NEJM AI, 1(12):AIcs2400639, 2024

    Jonathan Wang and Donald A Redelmeier. Cognitive biases and artificial intelligence.NEJM AI, 1(12):AIcs2400639, 2024

    work page 2024

  5. [5]

    Misinformation spreading on facebook.Complex spreading phenomena in social systems: Influence and contagion in real-world social networks, pages 177–196, 2018

    Fabiana Zollo and Walter Quattrociocchi. Misinformation spreading on facebook.Complex spreading phenomena in social systems: Influence and contagion in real-world social networks, pages 177–196, 2018

    work page 2018

  6. [6]

    Information overload, multi-tasking, and the socially networked jury: Why prosecutors should approach the media gingerly.J

    Andrew E Taslitz. Information overload, multi-tasking, and the socially networked jury: Why prosecutors should approach the media gingerly.J. Legal Prof., 37:89, 2012

    work page 2012

  7. [7]

    How do expectations shape perception? Trends in cognitive sciences, 22(9):764–779, 2018

    Floris P De Lange, Micha Heilbron, and Peter Kok. How do expectations shape perception? Trends in cognitive sciences, 22(9):764–779, 2018

    work page 2018

  8. [8]

    Default beliefs as a basis of social decision-making.Trends in Cognitive Sciences, 26(12):1026–1028, 2022

    Shihui Han. Default beliefs as a basis of social decision-making.Trends in Cognitive Sciences, 26(12):1026–1028, 2022

    work page 2022

Show all 57 references
  1. [9]

    Evaluating the moral beliefs encoded in llms.Advances in Neural Information Processing Systems, 36:51778–51809, 2023

    Nino Scherrer, Claudia Shi, Amir Feder, and David Blei. Evaluating the moral beliefs encoded in llms.Advances in Neural Information Processing Systems, 36:51778–51809, 2023

    2023

  2. [10]

    Arithmetic without algo- rithms: Language models solve math with a bag of heuristics.arXiv preprint arXiv:2410.21272, 2024

    Yaniv Nikankin, Anja Reusch, Aaron Mueller, and Yonatan Belinkov. Arithmetic without algo- rithms: Language models solve math with a bag of heuristics.arXiv preprint arXiv:2410.21272, 2024

    2024 arXiv

  3. [11]

    Responsible generative ai: A comprehensive study to explain llms

    Ishika Shruti, Amol Kumar, Arjun Seth, et al. Responsible generative ai: A comprehensive study to explain llms. In2024 International Conference on Electrical, Computer and Energy Technologies (ICECET, pages 1–6. IEEE, 2024

    2024

  4. [12]

    A unified approach to interpreting model predictions.arXiv preprint arXiv:1705.07874, 2017

    Scott Lundberg. A unified approach to interpreting model predictions.arXiv preprint arXiv:1705.07874, 2017

    2017 arXiv

  5. [13]

    Impossibility theorems for feature attribution.Proceedings of the National Academy of Sciences, 121(2):e2304406120, 2024

    Blair Bilodeau, Natasha Jaques, Pang Wei Koh, and Been Kim. Impossibility theorems for feature attribution.Proceedings of the National Academy of Sciences, 121(2):e2304406120, 2024

    2024

  6. [14]

    Predictive learning via rule ensembles.The Annals of Applied Statistics, pages 916–954, 2008

    Jerome H Friedman and Bogdan E Popescu. Predictive learning via rule ensembles.The Annals of Applied Statistics, pages 916–954, 2008

    2008

  7. [15]

    Shapg: new feature importance method based on the shapley value.arXiv preprint arXiv:2407.00506, 2024

    Chi Zhao, Jing Liu, and Elena Parilina. Shapg: new feature importance method based on the shapley value.arXiv preprint arXiv:2407.00506, 2024

    2024 arXiv

  8. [16]

    Rational shapley values

    David Watson. Rational shapley values. InProceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 1083–1094, 2022. 10

    2022

  9. [17]

    Explainable ai for trees: From local explanations to global understanding.arXiv preprint arXiv:1905.04610, 2019

    Scott M Lundberg, Gabriel Erion, Hugh Chen, Alex DeGrave, Jordan M Prutkin, Bala Nair, Ronit Katz, Jonathan Himmelfarb, Nisha Bansal, and Su-In Lee. Explainable ai for trees: From local explanations to global understanding.arXiv preprint arXiv:1905.04610, 2019

    1905 arXiv

  10. [18]

    Tokenshap: Interpreting large language models with monte carlo shapley value estimation.arXiv preprint arXiv:2407.10114, 2024

    Roni Goldshmidt and Miriam Horovicz. Tokenshap: Interpreting large language models with monte carlo shapley value estimation.arXiv preprint arXiv:2407.10114, 2024

    2024 arXiv

  11. [19]

    Concept-level explainability for auditing & steering llm responses.arXiv preprint arXiv:2505.07610, 2025

    Kenza Amara, Rita Sevastjanova, and Mennatallah El-Assady. Concept-level explainability for auditing & steering llm responses.arXiv preprint arXiv:2505.07610, 2025

    2025 arXiv

  12. [20]

    Transcoders find interpretable llm feature circuits.Advances in Neural Information Processing Systems, 37:24375–24410, 2024

    Jacob Dunefsky, Philippe Chlenski, and Neel Nanda. Transcoders find interpretable llm feature circuits.Advances in Neural Information Processing Systems, 37:24375–24410, 2024

    2024

  13. [21]

    Chatgpt: Literate or intelligent about un sustainable development goals?Plos one, 19(4):e0297521, 2024

    Raghu Raman, Hiran H Lathabai, Santanu Mandal, Payel Das, Tavleen Kaur, and Prema Nedungadi. Chatgpt: Literate or intelligent about un sustainable development goals?Plos one, 19(4):e0297521, 2024

    2024

  14. [22]

    Surveying attitudinal alignment between large language models vs

    Qingyang Wu, Ying Xu, Tingsong Xiao, Yunze Xiao, Yitong Li, Tianyang Wang, Yichi Zhang, Shanghai Zhong, Yuwei Zhang, Wei Lu, et al. Surveying attitudinal alignment between large language models vs. humans towards 17 sustainable development goals.arXiv preprint arXiv:2404.13885, 2024

    2024 arXiv

  15. [23]

    Decoding biases: Automated methods and llm judges for gender bias detection in language models.arXiv preprint arXiv:2408.03907, 2024

    Shachi H Kumar, Saurav Sahay, Sahisnu Mazumder, Eda Okur, Ramesh Manuvinakurike, Nicole Beckage, Hsuan Su, Hung-yi Lee, and Lama Nachman. Decoding biases: Automated methods and llm judges for gender bias detection in language models.arXiv preprint arXiv:2408.03907, 2024

    2024 arXiv

  16. [24]

    Benchmarking cognitive biases in large language models as evaluators.arXiv preprint arXiv:2309.17012, 2023

    Ryan Koo, Minhwa Lee, Vipul Raheja, Jong Inn Park, Zae Myung Kim, and Dongyeop Kang. Benchmarking cognitive biases in large language models as evaluators.arXiv preprint arXiv:2309.17012, 2023

    2023 arXiv

  17. [25]

    Model-agnostic interpretability of machine learning.arXiv preprint arXiv:1606.05386, 2016

    Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Model-agnostic interpretability of machine learning.arXiv preprint arXiv:1606.05386, 2016

    2016 arXiv

  18. [26]

    Addressing cognitive bias in medical language models.arXiv preprint arXiv:2402.08113, 2024

    Samuel Schmidgall, Carl Harris, Ime Essien, Daniel Olshvang, Tawsifur Rahman, Ji Woong Kim, Rojin Ziaei, Jason Eshraghian, Peter Abadir, and Rama Chellappa. Addressing cognitive bias in medical language models.arXiv preprint arXiv:2402.08113, 2024

    2024 arXiv

  19. [27]

    Is general-purpose ai reasoning sensitive to data-induced cognitive biases? dynamic benchmarking on typical software engineering dilemmas.arXiv preprint arXiv:2508.11278, 2025

    Francesco Sovrano, Gabriele Dominici, Rita Sevastjanova, Alessandra Stramiglio, and Alberto Bacchelli. Is general-purpose ai reasoning sensitive to data-induced cognitive biases? dynamic benchmarking on typical software engineering dilemmas.arXiv preprint arXiv:2508.11278, 2025

    2025

  20. [28]

    A semantic embedding space based on large language models for modelling human beliefs.Nature Human Behaviour, pages 1–13, 2025

    Byunghwee Lee, Rachith Aiyappa, Yong-Yeol Ahn, Haewoon Kwak, and Jisun An. A semantic embedding space based on large language models for modelling human beliefs.Nature Human Behaviour, pages 1–13, 2025

    2025

  21. [29]

    Brownian distance covariance.The Annals of Applied Statistics, pages 1236–1265, 2009

    Gábor J Székely and Maria L Rizzo. Brownian distance covariance.The Annals of Applied Statistics, pages 1236–1265, 2009

    2009

  22. [30]

    Dealing with information overload: a comprehensive review.Frontiers in psychology, 14:1122200, 2023

    Miriam Arnold, Mascha Goldschmitt, and Thomas Rigotti. Dealing with information overload: a comprehensive review.Frontiers in psychology, 14:1122200, 2023

    2023

  23. [31]

    Why information overload damages decisions? an explanation based on limited cognitive resources.Advances in Psychological Science, 27(10):1758, 2019

    Jingshang Che, Hailong Sun, Chenjie Xiao, and Aimei Li. Why information overload damages decisions? an explanation based on limited cognitive resources.Advances in Psychological Science, 27(10):1758, 2019

    2019

  24. [32]

    The power of moral words: Loaded language generates framing effects in the extreme dictator game.Judgment and Decision Making, 14(3):309–317, 2019

    Valerio Capraro and Andrea Vanzo. The power of moral words: Loaded language generates framing effects in the extreme dictator game.Judgment and Decision Making, 14(3):309–317, 2019

    2019

  25. [33]

    The fog index after twenty years.Journal of Business Communication, 6(2): 3–13, 1969

    Robert Gunning. The fog index after twenty years.Journal of Business Communication, 6(2): 3–13, 1969. 11

    1969

  26. [34]

    Judging the judges: Evaluating alignment and vulnerabilities in llms-as- judges.arXiv preprint arXiv:2406.12624, 2024

    Aman Singh Thakur, Kartik Choudhary, Venkat Srinik Ramayapally, Sankaran Vaidyanathan, and Dieuwke Hupkes. Judging the judges: Evaluating alignment and vulnerabilities in llms-as- judges.arXiv preprint arXiv:2406.12624, 2024

    2024 arXiv

  27. [35]

    Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in Neural Information Processing Systems, 36:46595–46623, 2023

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in Neural Information Processing Systems, 36:46595–46623, 2023

    2023

  28. [36]

    Shap for actuaries: Explain any model

    Michael Mayer, Daniel Meier, and Mario V Wuthrich. Shap for actuaries: Explain any model. Available at SSRN 4389797, 2023

    2023

  29. [37]

    shap.explainers.partition — shap documentation

    Scott Lundberg et al. shap.explainers.partition — shap documentation. https: //shap-community.readthedocs.io/en/latest/generated/shap.explainers. Partition.html, 2025. Accessed: 2025-09-22

    2025

  30. [38]

    Gradient boosting machines, a tutorial.Frontiers in neuro- robotics, 7:21, 2013

    Alexey Natekin and Alois Knoll. Gradient boosting machines, a tutorial.Frontiers in neuro- robotics, 7:21, 2013

    2013

  31. [39]

    Lasso regression.Journal of British Surgery, 105(10): 1348–1348, 2018

    Jonas Ranstam and Jonathan A Cook. Lasso regression.Journal of British Surgery, 105(10): 1348–1348, 2018

    2018

  32. [40]

    Xgboost: A scalable tree boosting system

    Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. InProceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pages 785–794, 2016

    2016

  33. [41]

    Sentence-bert: Sentence embeddings using siamese bert- networks

    Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert- networks. InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11 2019. URL https://arxiv.org/ abs/1908.10084

    2019 arXiv

  34. [42]

    Glocalx-from local to global explanations of black box ai models.Artificial Intelligence, 294:103457, 2021

    Mattia Setzu, Riccardo Guidotti, Anna Monreale, Franco Turini, Dino Pedreschi, and Fosca Gi- annotti. Glocalx-from local to global explanations of black box ai models.Artificial Intelligence, 294:103457, 2021

    2021

  35. [43]

    imodels: a python package for fitting interpretable models.Journal of open source software, 6(61):3192, 2021

    Chandan Singh, Keyan Nasseri, Yan Shuo Tan, Tiffany Tang, and Bin Yu. imodels: a python package for fitting interpretable models.Journal of open source software, 6(61):3192, 2021

    2021

  36. [44]

    Bayesian rule sets for interpretable classification

    Tong Wang, Cynthia Rudin, Finale Velez-Doshi, Yimin Liu, Erica Klampfl, and Perry MacNeille. Bayesian rule sets for interpretable classification. In2016 IEEE 16th International Conference on Data Mining (ICDM), pages 1269–1274, 2016. doi: 10.1109/ICDM.2016.0171

    2016

  37. [45]

    Fast interpretable greedy-tree sums (figs)

    Yan Shuo Tan, Chandan Singh, Keyan Nasseri, Abhineet Agarwal, James Duncan, Omer Ronen, Matthew Epland, Aaron Kornblith, and Bin Yu. Fast interpretable greedy-tree sums (figs). ArXivorg, 2023

    2023

  38. [46]

    Post-hoc explanation using a mimic rule for numerical data

    Kohei Asano and Jinhee Chun. Post-hoc explanation using a mimic rule for numerical data. InProceedings of the 13th International Conference on Agents and Artificial Intelligence. SCITEPRESS-Science and Technology Publications, 2021

    2021

  39. [47]

    Palm: Machine learning explanations for iterative debugging

    Sanjay Krishnan and Eugene Wu. Palm: Machine learning explanations for iterative debugging. InProceedings of the 2Nd workshop on human-in-the-loop data analytics, pages 1–6, 2017

    2017

  40. [48]

    V oorhees

    Ellen M. V oorhees. The TREC-8 question answering track report. In Ellen M. V oorhees and Donna K. Harman, editors,Proceedings of The Eighth Text REtrieval Conference, TREC 1999, Gaithersburg, Maryland, USA, November 17-19, 1999, volume 500-246 ofNIST Special Publication. Nati...

    1999

  41. [49]

    Geni: A framework for the generation of explanations and insights of knowledge graph embedding predic- tions.Neurocomputing, 521:199–212, 2023

    Elvira Amador-Domínguez, Emilio Serrano, and Daniel Manrique. Geni: A framework for the generation of explanations and insights of knowledge graph embedding predic- tions.Neurocomputing, 521:199–212, 2023. doi: 10.1016/J.NEUCOM.2022.12.010. URL https://doi.org/10.1016/j.neucom...

    2023 doi

  42. [50]

    Vera Liao, Yunfeng Zhang, Ronny Luss, Finale Doshi-Velez, and Amit Dhurandhar

    Q. Vera Liao, Yunfeng Zhang, Ronny Luss, Finale Doshi-Velez, and Amit Dhurandhar. Con- necting algorithmic research and usage contexts: A perspective of contextualized evaluation for explainable AI. In Jane Hsu and Ming Yin, editors,Proceedings of the Tenth AAAI Conference on ...

    2022 doi

  43. [51]

    From anecdotal evidence to quantitative evaluation methods: A systematic review on evaluating explainable AI.ACM Comput

    Meike Nauta, Jan Trienes, Shreyasi Pathak, Elisa Nguyen, Michelle Peters, Yasmin Schmitt, Jörg Schlötterer, Maurice van Keulen, and Christin Seifert. From anecdotal evidence to quantitative evaluation methods: A systematic review on evaluating explainable AI.ACM Comput. Surv.,...

    2023 doi

  44. [52]

    Notions of explainability and evaluation approaches for explainable artificial intelligence.Information Fusion, 76:89–106, 2021

    Giulia Vilone and Luca Longo. Notions of explainability and evaluation approaches for explainable artificial intelligence.Information Fusion, 76:89–106, 2021

    2021

  45. [53]

    When do neural nets outperform boosted trees on tabular data?Advances in Neural Information Processing Systems, 36:76336–76369, 2023

    Duncan McElfresh, Sujay Khandagale, Jonathan Valverde, Vishak Prasad C, Ganesh Ramakr- ishnan, Micah Goldblum, and Colin White. When do neural nets outperform boosted trees on tabular data?Advances in Neural Information Processing Systems, 36:76336–76369, 2023

    2023

  46. [54]

    textstat: Python library for readability statistics

    textstat. textstat: Python library for readability statistics. https://pypi.org/project/ textstat/, 2025. Version 0.7.10

    2025

  47. [55]

    Evaluate the conceptual density of the texts in the whole web about {topic}. Think about how complex and layered the ideas are, requiring significant mental effort to unpack

    Taha Yasseri, András Kornai, and János Kertész. A practical approach to language complexity: a wikipedia case study.PloS one, 7(11):e48386, 2012. 13 A Input Space Abstractions Below, we provide expanded definitions and illustrative examples for each of the input properties map...

    2012

  48. [56]

    Topic 1 label: a very short explanation of why it’s score 17 {score}

  49. [57]

    Write␣one␣short␣sentence

    Topic 2 label: short explanation of why score {score}; ... During generation, the samplingtemperaturewas set to T= 1 , andtop_pwas also set to 1 to encourage sufficient diversity in the topics. D Bias Injection Procedure Although the XAI methods we evaluate (e.g., SHAP) rely o...

    2005

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.