Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

The paper argues that a local LLM explanation is faithful only if the model also answers causal, contrastive, and counterfactual stress-test prompts correctly.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 09:05 UTC pith:EZ2IYVQ6

load-bearing objection A useful survey with a genuinely new granularity taxonomy; the faithfulness-via-stress-test claim is asserted, not argued, and the supporting experiment is too thin to carry it. the 3 major comments →

arxiv 2510.17256 v2 pith:EZ2IYVQ6 submitted 2025-10-20 cs.CL

Explainability of Large Language Models: Opportunities and Challenges toward Generating Trustworthy Explanations

Shahin Atakishiyev , Housam K.B. Babiker , Jiayi Dai , Nawshad Farruque , Teruaki Hayashi , Nafisa Sadaf Hriti , Md Abed Rahman , Iain Smith
show 3 more authors
Mi-Young Kim Osmar R. Za\"iane Randy Goebel
This is my paper
classification cs.CL
keywords local explainabilitymechanistic interpretabilityLLM trustworthinessstress-test faithfulnesschain-of-thoughtcounterfactual reasoningexplanation granularitysafety-critical AI
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that trust in large language models requires two complementary kinds of understanding: local explanations for individual decisions and mechanistic explanations of the system's inner workings. Its central proposal is a practical test for faithful local explanations: an explanation is faithful only if the model also predicts correct answers to causal, contrastive, and counterfactual stress-test questions about the same situation. The paper draws this test from an experiment in autonomous driving, where a multimodal model passed the test in three of four scenes and failed in one. It also shows that the same failure of faithfulness appears in healthcare, where self-explanations from language models can sound plausible while being wrong. To make explanations usable, the paper proposes three levels of granularity matched to different audiences and ties trustworthy explanations to eight principles of safe, truthful, fair, robust, private, ethical, transparent, and accountable AI.

Core claim

The paper's core claim is that a local explanation is faithful if the model also predicts correct answers in stress test prompts — causal questions, contrastive questions, and counterfactual questions posed as follow-ups to the original explanation. Applying this test with a multimodal model on four driving scenes, the authors find that the model produces correct responses in three scenes and fails in one, where it first gives an incorrect explanation and then cannot justify a contrastive query. From this they derive four properties that trustworthy local explanations should satisfy: passing the causal reasoning test, justifying contrastive and causal follow-ups, distinguishing fact from bel

What carries the argument

The central object is the stress-test definition of local explanation faithfulness: an explanation is faithful if the model survives follow-up causal, contrastive, and counterfactual prompts. This converts an abstract epistemic property into an operational test. The paper's other machinery is a three-level granularity taxonomy — coarse, coarse-to-fine, and fine-grained explanations — matched to four explainee groups (general users, regulators, domain experts, developers), and an alignment of explanation requirements with eight trust principles (safety, truthfulness, fairness, robustness, privacy, machine ethics, transparency, accountability).

Load-bearing premise

The paper's standard for trustworthy explanations rests on equating faithfulness with answering stress-test questions correctly; if a model can pass such follow-ups while its stated reasons do not match its actual internal computation, the standard collapses.

What would settle it

Take a model that passes causal, contrastive, and counterfactual stress tests on a set of scenes, then use activation patching or circuit analysis on the same scenes to check whether the tokens appearing in the explanation are the ones causally responsible for the output. If the explanation cites features that patching shows are irrelevant, the stress-test standard would declare the explanation faithful while mechanistic analysis shows it is not.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the stress-test standard holds, explanation quality can be measured directly by posing follow-up questions, rather than by judging whether the stated reasons match internal computations.
  • Trustworthy local explanations must include the model's refusal behavior: acknowledging when it cannot answer is as important as giving a correct explanation.
  • The three-level granularity scheme means explanations should be adapted to the explainee: coarse for general users and regulators, coarse-to-fine for domain experts, and fine-grained for developers.
  • Local and mechanistic explanations are complementary; a trustworthy LLM needs both decision-level justification and system-level understanding.
  • In safety-critical domains, LLM self-explanations cannot be taken as faithful by default, because they are generated by next-token prediction without explicit control over the reasoning process.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The stress-test standard could be extended to other domains: a faithful explanation of a medical diagnosis would require correct answers to 'what if' and 'why not' questions about the patient's case.
  • If passing stress tests becomes a release criterion, models will be actively tuned to answer counterfactuals, potentially improving causal reasoning as a side effect.
  • The operational definition conflates explanation faithfulness with predictive robustness; a model could pass all stress tests while its stated reasons still misdescribe its internal computation, a failure mode the paper itself documents for chain-of-thought reasoning.
  • The granularity taxonomy suggests a calibration task: determine which granularity level each explainee actually needs, and measure whether explanations at that level change their trust or behavior.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper is a survey-plus-position manuscript on explainability of large language models. It reviews local explainability (natural language explanations, chain-of-thought, retrieval-augmented generation, feature attribution) and mechanistic interpretability (circuits, attention heads, activation patching, logit probing), presents two small empirical case studies in healthcare and autonomous driving, and argues that trustworthy local explanations must satisfy four properties, including passing causal/contrastive/counterfactual stress tests. It also proposes a three-level granularity scheme for explanations and maps explanation requirements to the eight TrustLLM principles.

Significance. The survey portions are comprehensive and well organized: the tables of representative studies, the distinction between local and mechanistic explanation, and the discussion of fact-vs-belief and tacit-vs-explicit knowledge provide a useful synthesis for researchers entering the field. The paper also honestly acknowledges that chain-of-thought explanations are often unfaithful (§4.2, citing [55,56]) and that LLMs lack self-representation (§4.3). If the proposed stress-test-based definition of faithful local explanations were valid, it would give practitioners a concrete operational checklist. However, that central normative contribution is not supported by the evidence or argument presented, and the paper's own cited findings on unfaithful CoT undercut it. The survey value alone is moderate; the position claim needs substantial revision.

major comments (3)
  1. [§7.2, Figure 8] The load-bearing assertion equates faithfulness with passing causal/contrastive/counterfactual stress-test follow-ups. No argument is given that behavioral consistency on such prompts entails fidelity to the model's internal decision process. The paper itself states in §7.1 that 'the faithfulness of an LLM's explanations cannot be fulfilled by simply asking the LLM itself to explain' and cites in §4.2 evidence that CoT can be plausible yet unfaithful ([55,56]). Since the follow-up answers are generated by the same auto-regressive next-token predictor, a model can produce coherent, self-consistent justifications that pass the proposed stress test while misrepresenting the actual computation. Thus the four-property checklist in §8.2 is at best a test of self-consistency, not of faithfulness to the model's reasoning. The paper needs either a substantive argument against this failure mode or
  2. [§7.2, Figure 8] The empirical support for the proposed definition is one multimodal LLM (Video-LLaMA2) on four hand-sampled BDD-A scenes, with manual judgment that responses are 'correct.' There is no inter-rater reliability, no quantitative metric, no baseline or comparison model, and no error analysis. This is far too thin to validate a new definition of faithfulness and to support the general claim that a local explanation is trustworthy if it satisfies the four properties. The authors should either provide a much larger, public-benchmark-based evaluation with statistical reporting, or explicitly weaken the claim from a definition to a hypothesis with clear validation requirements.
  3. [§7.1] The healthcare discussion argues that rationale extraction 'guarantees that the selected features are truly the reason for the decision-making' because the classifier is forced to predict from the selected subset. This is true only for the select-predict architecture, not for the original model being explained; it is a different notion of faithfulness than the one applied to LLM self-explanations. The paper's own criticism of LIME as an approximation would apply equally to any surrogate-like explanation. Moreover, the justification for distrusting LLM self-explanations—that there is no explicit control over the inference process—also applies to the §7.2 stress-test definition, which introduces no such control. The internal tension between the healthcare argument and the autonomous-driving definition should be resolved.
minor comments (4)
  1. [§6.2] Typo: 'Automation evaluation' should be 'Automatic evaluation'; also 'area under curve (AOC)' should be 'AUC'.
  2. [§5.1] Misspelling: 'Akkra' should be 'Accra'.
  3. [§8.2] Property 2 ('justify contrastive and causal questions correctly') overlaps with property 1 ('passes the causal reasoning test'); consider rewording to clarify the distinct contribution of each property.
  4. [References] Some reference entries are inconsistently formatted, e.g., author lists such as 'Kimi Team, Angang Du, Bofei Gao...' and missing URLs for some preprints. Please harmonize the style.

Circularity Check

1 steps flagged

Stress-test faithfulness criterion is stipulated in §7.2 and then restated as the four-property checklist; this is a minor definitional circularity, not a fitted prediction or load-bearing self-citation chain.

specific steps
  1. self definitional [Section 7.2 (Insights from Autonomous Driving), Figure 8; reified in Section 8.2]
    "However, we argue that a local explanation is faithful if the model also predicts correct answers in stress test prompts. ... Based on these insights, we can summarize that a local LLM explanation in this example can be considered trustworthy if 1) it passes the causal reasoning test, 2) it can justify contrastive and causal questions correctly as a stress test, 3) it understands, distinguishes, and justify questions requiring fact or belief-based subjective answers accordingly, and 4) it avoids presenting falsified explanation in case it just cannot answer the question."

    The four-property trustworthiness checklist is a restatement of the paper's own stipulated definition: faithfulness was defined as correct stress-test answering, and properties 1 and 2 are exactly that stress-test success unpacked. The Figure 8 demonstrations are interpreted through this criterion rather than independently testing it, so the 'insights' cannot validate the definition. The step is definitional rather than an equation fitted to data, which keeps the severity low.

full rationale

This is primarily a scoping review and position paper. Its formal content—the LIME objective, mechanistic mapping, transformer equations—is borrowed from the literature and is not used to derive new predictions, so nothing reduces to fitted quantities. The only derivation-like step is the §7.2 proposal that stress-test success defines faithful local explanations; the subsequent four-property checklist then restates that stipulation. I flag this as a minor self-definitional move rather than a fitted-input circularity because the paper explicitly says 'we argue', making the criterion a proposed definition rather than a hidden empirical inference. The healthcare discussion does use the authors' own rationale-extraction example from [159, 163, 164], but the load-bearing guarantee of select-predict architectures comes from the externally cited Lei et al. design and the figure is illustrative, so there is no self-citation chain forcing the result. The other contributions—taxonomies, granularity levels, literature synthesis, and the complementarity claim—are self-contained and do not reduce to their inputs. Score 2 reflects the single definitional circularity in the normative stress-test framework, with the rest of the paper independently grounded.

Axiom & Free-Parameter Ledger

0 free parameters · 5 axioms · 0 invented entities

Survey paper: no parameters are fitted and no new entities are postulated. The central claim's load-bearing inputs are definitional and normative. (1) Faithfulness is operationally equated with passing causal/contrastive/counterfactual stress-test Q&A — an ad hoc premise introduced in §7.2 that the entire trustworthiness checklist rests on. (2) CoT is treated as a local explanation even though the paper cites a position that CoT is not explainability. (3) The eight TrustLLM principles are adopted without independent justification as the normative gold standard. (4) The claim that transformer models lack self-representation is asserted without a citation and is then used to argue self-explanations cannot be faithful. The single standard-math input is the attention equation (1) from Vaswani et al.

axioms (5)
  • ad hoc to paper Faithfulness of a local explanation is equivalent to the model answering causal/contrastive/counterfactual stress-test questions correctly
    Introduced in §7.2 without citation or derivation: 'we argue that a local explanation is faithful if the model also predicts correct answers in stress test prompts.' All four normative properties in §8.2 hang on it.
  • domain assumption Chain-of-thought reasoning qualifies as a local explanation
    §4.2 categorizes CoT as a local explanation while the paper also cites [89] 'Chain-of-Thought Is Not Explainability'; the classification is assumed for the survey's frame.
  • domain assumption TrustLLM's eight principles are the correct normative frame for trustworthy LLM explanations
    §8.3 adopts the eight principles (safety, truthfulness, fairness, robustness, privacy, machine ethics, transparency, accountability) from [75] without independent justification and organizes all trust claims on them.
  • domain assumption Transformer-based foundation models have no capacity for self-representation
    Asserted in §4.3 and repeated in §7.1 without citation; used to conclude that self-explanations cannot be verified and thus cannot be guaranteed faithful.
  • standard math Self-attention computation of Eq. (1) correctly models the transformer's internal computation
    Standard Vaswani et al. [1] attention; the grounding for all mechanistic interpretation described in §4.3.

pith-pipeline@v1.3.0-alltime-deepseek · 33413 in / 18693 out tokens · 146558 ms · 2026-08-04T09:05:18.191963+00:00 · methodology

0 comments
read the original abstract

Large language models have exhibited impressive performance across a broad range of downstream tasks in natural language processing. However, how a language model predicts the next token and generates content is not generally understandable by humans. Furthermore, these models often make errors in prediction and reasoning, known as hallucinations. These errors underscore the urgent need to better understand and interpret the intricate inner workings of language models and how they generate predictive outputs. Motivated by this gap, this paper investigates local explainability and mechanistic interpretability within Transformer-based large language models to foster trust in such models. In this regard, our paper aims to make three key contributions. First, we present a review of local explainability and mechanistic interpretability approaches and insights from relevant studies in the literature. Furthermore, we describe experimental studies on explainability and reasoning with large language models in two critical domains -- healthcare and autonomous driving -- and analyze the trust implications of such explanations for explanation receivers. Finally, we summarize current unaddressed issues in the evolving landscape of LLM explainability and outline the opportunities, critical challenges, and future directions toward generating human-aligned, trustworthy LLM explanations.

Figures

Figures reproduced from arXiv: 2510.17256 by Housam K.B. Babiker, Iain Smith, Jiayi Dai, Md Abed Rahman, Mi-Young Kim, Nafisa Sadaf Hriti, Nawshad Farruque, Osmar R. Za\"iane, Randy Goebel, Shahin Atakishiyev, Teruaki Hayashi.

Figure 1
Figure 1. Figure 1: Local explainability and mechanistic interpretability approaches in LLMs. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The architecture of Transformer. Source: [ [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: An example of an induction circuit provided by Rai et al. [ [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: ChatGPT provides a diagnostic prediction and the corresponding evidence when provided [PITH_FULL_IMAGE:figures/full_fig_p014_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: An example of a serious hallucination of GPT 3.5 in clinical diagnosis on the Med-HALT [PITH_FULL_IMAGE:figures/full_fig_p015_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Running LIME on classifying a medical note with BERT. [PITH_FULL_IMAGE:figures/full_fig_p016_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Rationale extraction selects a subset of features as a rationale for a diagnostic prediction on [PITH_FULL_IMAGE:figures/full_fig_p017_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Scene understanding with the Video-LLaMA2 multimodal LLM: We first ask the model to [PITH_FULL_IMAGE:figures/full_fig_p019_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Granularity of LLM explanations with respect to the explainees [PITH_FULL_IMAGE:figures/full_fig_p020_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: The eight principles of a human-centered trustworthy LLM and the role of LLM explana [PITH_FULL_IMAGE:figures/full_fig_p021_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Evaluating the Impact of Explainable AI on Trust in AI-Assisted Code Review

    cs.SE 2026-07 conditional novelty 6.0

    Fuller XAI raises perceived trust in AI code review, while moderate explanations maximize agreement; no explanations minimize both, with no significant time effect.

Reference graph

Works this paper leans on

205 extracted references · 41 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Attention Is All You Need.Advances in Neural Information Processing Systems, 2017

    A Vaswani. Attention Is All You Need.Advances in Neural Information Processing Systems, 2017

    2017

  2. [2]

    Language models are few-shot learners.Advances in Neural Information Processing Systems, 33:1877– 1901, 2020

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners.Advances in Neural Information Processing Systems, 33:1877– 1901, 2020

    1901

  3. [3]

    GPT-4 Technical Report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. GPT-4 Technical Report.arXiv preprint arXiv:2303.08774, 2023

    Pith/arXiv arXiv 2023

  4. [4]

    GPT-5 System Card, August 2025

    OpenAI. GPT-5 System Card, August 2025

    2025

  5. [5]

    Gonzalez, Ion Stoica, and Eric P

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90%* ChatGPT Quality, March 2023

    2023

  6. [6]

    LLaMA: Open and Efficient Foundation Language Models.arXiv preprint arXiv:2302.13971, 2023

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Tim- othée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. LLaMA: Open and Efficient Foundation Language Models.arXiv preprint arXiv:2302.13971, 2023

    Pith/arXiv arXiv 2023

  7. [7]

    Llama 2: Open Foundation and Fine-Tuned Chat Models .arXiv preprint arXiv:2307.09288, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open Foundation and Fine-Tuned Chat Models .arXiv preprint arXiv:2307.09288, 2023

    Pith/arXiv arXiv 2023

  8. [8]

    Mistral 7b.arXiv preprint arXiv:2310.06825, 2023

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b.arXiv preprint arXiv:2310.06825, 2023

    Pith/arXiv arXiv 2023

  9. [9]

    Introducing Claude

    Anthropic. Introducing Claude. https://www.anthropic.com/news/introducing-claude, 2023

    2023

  10. [10]

    Introducing Claude Sonnet 4.5, September 2025

    Anthropic. Introducing Claude Sonnet 4.5, September 2025

    2025

  11. [11]

    Qwen technical report.arXiv preprint arXiv:2309.16609, 2023

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report.arXiv preprint arXiv:2309.16609, 2023

    Pith/arXiv arXiv 2023

  12. [12]

    Qwen2 Technical Report.arXiv preprint arXiv:2407.10671, 2024

    Qwen Team. Qwen2 Technical Report.arXiv preprint arXiv:2407.10671, 2024

    Pith/arXiv arXiv 2024

  13. [13]

    Qwen3 Technical Report.arXiv preprint arXiv:2505.09388, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 Technical Report.arXiv preprint arXiv:2505.09388, 2025

    Pith/arXiv arXiv 2025

  14. [14]

    Gemini: A Family of Highly Capable Multimodal Models.arXiv preprint arXiv:2312.11805, 2023

    Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. Gemini: A Family of Highly Capable Multimodal Models.arXiv preprint arXiv:2312.11805, 2023

    Pith/arXiv arXiv 2023

  15. [15]

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025

    Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025

    Pith/arXiv arXiv 2025

  16. [16]

    DeepSeek LLM: Scaling Open-Source Language Models with Longtermism.arXiv preprint arXiv:2401.02954, 2024

    Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, et al. DeepSeek LLM: Scaling Open-Source Language Models with Longtermism.arXiv preprint arXiv:2401.02954, 2024. 23

    Pith/arXiv arXiv 2024

  17. [17]

    Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model.arXiv preprint arXiv:2405.04434, 2024

    Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, et al. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model.arXiv preprint arXiv:2405.04434, 2024

    Pith/arXiv arXiv 2024

  18. [18]

    Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    Pith/arXiv arXiv 2024

  19. [19]

    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning .arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning .arXiv preprint arXiv:2501.12948, 2025

    Pith/arXiv arXiv 2025

  20. [20]

    Open Release of Grok-1.https://x.ai/news/grok-os, 2024

    xAI. Open Release of Grok-1.https://x.ai/news/grok-os, 2024

    2024

  21. [21]

    Grok-2 beta release.https://x.ai/news/grok-2, 2024

    xAI. Grok-2 beta release.https://x.ai/news/grok-2, 2024

    2024

  22. [22]

    Grok 4.https://x.ai/news/grok-4, 2025

    xAI. Grok 4.https://x.ai/news/grok-4, 2025

    2025

  23. [23]

    Kimi k1.5: Scaling Reinforcement Learning with LLMs.arXiv preprint arXiv:2501.12599, 2025

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1.5: Scaling Reinforcement Learning with LLMs.arXiv preprint arXiv:2501.12599, 2025

    Pith/arXiv arXiv 2025

  24. [24]

    Kimi k2: Open agentic intelligence.arXiv preprint arXiv:2507.20534, 2025

    Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, et al. Kimi k2: Open agentic intelligence.arXiv preprint arXiv:2507.20534, 2025

    Pith/arXiv arXiv 2025

  25. [25]

    PaLM-E: An Embodied Multimodal Language Model

    Danny Driess, Fei Xia, Mehdi SM Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, et al. PaLM-E: An Embodied Multimodal Language Model. InInternational Conference on Machine Learning, pages 8469–8488. PMLR, 2023

    2023

  26. [26]

    Emergent Abilities of Large Language Models.Transactions on Machine Learning Research, 2022

    Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. Emergent Abilities of Large Language Models.Transactions on Machine Learning Research, 2022

    2022

  27. [27]

    Rethink- ing Interpretability in the Era of Large Language Models.arXiv preprint arXiv:2402.01761, 2024

    Chandan Singh, Jeevana Priya Inala, Michel Galley, Rich Caruana, and Jianfeng Gao. Rethink- ing Interpretability in the Era of Large Language Models.arXiv preprint arXiv:2402.01761, 2024

    Pith/arXiv arXiv 2024

  28. [28]

    A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions.ACM Transactions on Information Systems, 43(2):1–55, 2025

    Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qian- glong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions.ACM Transactions on Information Systems, 43(2):1–55, 2025

    2025

  29. [29]

    Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations.arXiv preprint arXiv:2504.12691, 2025

    Yiyou Sun, Yu Gai, Lijie Chen, Abhilasha Ravichander, Yejin Choi, and Dawn Song. Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations.arXiv preprint arXiv:2504.12691, 2025

    Pith/arXiv arXiv 2025

  30. [30]

    OpinionGPT: Modelling Explicit Biases in Instruction-Tuned LLMs.arXiv preprint arXiv:2309.03876, 2023

    Patrick Haller, Ansar Aynetdinov, and Alan Akbik. OpinionGPT: Modelling Explicit Biases in Instruction-Tuned LLMs.arXiv preprint arXiv:2309.03876, 2023

    Pith/arXiv arXiv 2023

  31. [31]

    Artificial Intelligence, Values, and Alignment.Minds and Machines, 30(3):411– 437, 2020

    Iason Gabriel. Artificial Intelligence, Values, and Alignment.Minds and Machines, 30(3):411– 437, 2020

    2020

  32. [32]

    A Multi-Component Framework for the Analysis and Design of Explainable Artificial Intelligence.Machine Learning and Knowledge Extraction, 3(4):900– 921, 2021

    Mi-Young Kim, Shahin Atakishiyev, Housam Khalifa Bashier Babiker, Nawshad Farruque, Randy Goebel, Osmar R Zaïane, Mohammad-Hossein Motallebi, Juliano Rabelo, Talat Syed, Hengshuai Yao, et al. A Multi-Component Framework for the Analysis and Design of Explainable Artificial Intelligence.Machine Learning and Knowledge Extraction, 3(4):900– 921, 2021

    2021

  33. [33]

    Explainable AI (XAI): A systematic meta-survey of current challenges and future opportunities.Knowledge-Based Systems, 263:110273, 2023

    Waddah Saeed and Christian Omlin. Explainable AI (XAI): A systematic meta-survey of current challenges and future opportunities.Knowledge-Based Systems, 263:110273, 2023. 24

    2023

  34. [34]

    Class-based n-gram models of natural language.Computational linguistics, 18(4):467–480, 1992

    Peter F Brown, Vincent J Della Pietra, Peter V Desouza, Jennifer C Lai, and Robert L Mercer. Class-based n-gram models of natural language.Computational linguistics, 18(4):467–480, 1992

    1992

  35. [35]

    On structuring probabilistic dependences in stochastic language modelling.Computer Speech & Language, 8(1):1–38, 1994

    Hermann Ney, Ute Essen, and Reinhard Kneser. On structuring probabilistic dependences in stochastic language modelling.Computer Speech & Language, 8(1):1–38, 1994

    1994

  36. [36]

    Learning representations by back-propagating errors.Nature, 323(6088):533–536, 1986

    David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning representations by back-propagating errors.Nature, 323(6088):533–536, 1986

    1986

  37. [37]

    A Neural Probabilistic Language Model .Advances in Neural Information Processing Systems, 13, 2000

    Yoshua Bengio, Réjean Ducharme, and Pascal Vincent. A Neural Probabilistic Language Model .Advances in Neural Information Processing Systems, 13, 2000

    2000

  38. [38]

    Distributed repre- sentations of words and phrases and their compositionality.Advances in Neural Information Processing Systems, 26, 2013

    Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed repre- sentations of words and phrases and their compositionality.Advances in Neural Information Processing Systems, 26, 2013

    2013

  39. [39]

    Enriching word vectors with subword information.Transactions of the Association for Computational Linguistics, 5:135–146, 2017

    Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. Enriching word vectors with subword information.Transactions of the Association for Computational Linguistics, 5:135–146, 2017

    2017

  40. [40]

    Retrofitting word vectors to semantic lexicons.arXiv preprint arXiv:1411.4166, 2014

    Manaal Faruqui, Jesse Dodge, Sujay K Jauhar, Chris Dyer, Eduard Hovy, and Noah A Smith. Retrofitting word vectors to semantic lexicons.arXiv preprint arXiv:1411.4166, 2014

    Pith/arXiv arXiv 2014

  41. [41]

    Augmenting semantic representation of depressive language: From forums to microblogs

    Nawshad Farruque, Osmar Zaiane, and Randy Goebel. Augmenting semantic representation of depressive language: From forums to microblogs. InJoint European conference on machine learning and knowledge discovery in databases, pages 359–375. Springer, 2019

    2019

  42. [42]

    Universal sentence encoder for english

    Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, et al. Universal sentence encoder for english. InProceedings of the 2018 conference on empirical methods in natural language processing: system demonstrations, pages 169–174, 2018

    2018

  43. [43]

    Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer

    Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. Deep contextualized word representations.CoRR, abs/1802.05365, 2018

    Pith/arXiv arXiv 2018

  44. [44]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. InProceedings of naacL-HLT, volume 1, page 2. Minneapolis, Minnesota, 2019

    2019

  45. [45]

    ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

    Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. InInternational Conference on Learning Representations, 2020

    2020

  46. [46]

    RoBERTa: A Robustly Optimized BERT Pretraining Approach.arXiv preprint arXiv:1907.11692, 2019

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. RoBERTa: A Robustly Optimized BERT Pretraining Approach.arXiv preprint arXiv:1907.11692, 2019

    Pith/arXiv arXiv 1907

  47. [47]

    A brief overview of chatgpt: The history, status quo and potential future development.IEEE/CAA Journal of Automatica Sinica, 10(5):1122–1136, 2023

    Tianyu Wu, Shizhu He, Jingping Liu, Siqi Sun, Kang Liu, Qing-Long Han, and Yang Tang. A brief overview of chatgpt: The history, status quo and potential future development.IEEE/CAA Journal of Automatica Sinica, 10(5):1122–1136, 2023

    2023

  48. [48]

    BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

    Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. InProceed- ings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871–7880, 2020

    2020

  49. [49]

    Introducing ChatGPT, 2022

    OpenAI. Introducing ChatGPT, 2022

    2022

  50. [50]

    RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs.ACM Computing Surveys, 58(2):1–37, 2025

    Shreyas Chaudhari, Pranjal Aggarwal, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, Karthik Narasimhan, Ameet Deshpande, and Bruno Castro da Silva. RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs.ACM Computing Surveys, 58(2):1–37, 2025. 25

    2025

  51. [51]

    GPT-4o System Card.arXiv preprint arXiv:2410.21276, 2024

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. GPT-4o System Card.arXiv preprint arXiv:2410.21276, 2024

    Pith/arXiv arXiv 2024

  52. [52]

    Retrieval- Augmented Generation for Knowledge-Intensive NLP Tasks.Advances in Neural Information Processing Systems, 33:9459–9474, 2020

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Na- man Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval- Augmented Generation for Knowledge-Intensive NLP Tasks.Advances in Neural Information Processing Systems, 33:9459–9474, 2020

    2020

  53. [53]

    Benchmarking Large Language Models in Retrieval-Augmented Generation

    Jiawei Chen, Hongyu Lin, Xianpei Han, and Le Sun. Benchmarking Large Language Models in Retrieval-Augmented Generation. InProceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 17754–17762, 2024

    2024

  54. [54]

    Enhancing ai systems with agentic workflows patterns in large language model

    Aditi Singh, Abul Ehtesham, Saket Kumar, and Tala Talaei Khoei. Enhancing ai systems with agentic workflows patterns in large language model. In2024 IEEE World AI IoT Congress (AIIoT), pages 527–532. IEEE, 2024

    2024

  55. [55]

    Language Models Don’t Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting

    Miles Turpin, Julian Michael, Ethan Perez, and Samuel Bowman. Language Models Don’t Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting. Advances in Neural Information Processing Systems, 36:74952–74965, 2023

    2023

  56. [56]

    Chain-of-thought reasoning in the wild is not always faithful.arXiv preprint arXiv:2503.08679, 2025

    Iván Arcuschin, Jett Janiak, Robert Krzyzanowski, Senthooran Rajamanoharan, Neel Nanda, and Arthur Conmy. Chain-of-thought reasoning in the wild is not always faithful.arXiv preprint arXiv:2503.08679, 2025

    Pith/arXiv arXiv 2025

  57. [57]

    Explainability for Large Language Models: A Survey

    Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du. Explainability for Large Language Models: A Survey. ACM Transactions on Intelligent Systems and Technology, 15(2):1–38, 2024

    2024

  58. [58]

    A Unified Approach to Interpreting Model Predictions

    Scott M Lundberg and Su-In Lee. A Unified Approach to Interpreting Model Predictions . Advances in Neural Information Processing Systems, 30, 2017

    2017

  59. [59]

    From Understanding to Utilization: A Survey on Explainability for Large Language Models.arXiv preprint arXiv:2401.12874, 2024

    Haoyan Luo and Lucia Specia. From Understanding to Utilization: A Survey on Explainability for Large Language Models.arXiv preprint arXiv:2401.12874, 2024

    Pith/arXiv arXiv 2024

  60. [60]

    Large Language Models and Causal Inference in Collaboration: A Comprehensive Survey.arXiv preprint arXiv:2403.09606, 2024

    Xiaoyu Liu, Paiheng Xu, Junda Wu, Jiaxin Yuan, Yifan Yang, Yuhang Zhou, Fuxiao Liu, Tianrui Guan, Haoliang Wang, Tong Yu, et al. Large Language Models and Causal Inference in Collaboration: A Comprehensive Survey.arXiv preprint arXiv:2403.09606, 2024

    Pith/arXiv arXiv 2024

  61. [61]

    Local Explanations for Large Language Models: a Brief Review of Methods

    Egor N V olkov and Alexey N Averkin. Local Explanations for Large Language Models: a Brief Review of Methods. In2024 XXVII International Conference on Soft Computing and Measurements (SCM), pages 189–192. IEEE, 2024

    2024

  62. [62]

    A Primer on the Inner Workings of Transformer-based Language Models.arXiv preprint arXiv:2405.00208, 2024

    Javier Ferrando, Gabriele Sarti, Arianna Bisazza, and Marta R Costa-Jussà. A Primer on the Inner Workings of Transformer-based Language Models.arXiv preprint arXiv:2405.00208, 2024

    Pith/arXiv arXiv 2024

  63. [63]

    A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models.arXiv preprint arXiv:2407.02646, 2024

    Daking Rai, Yilun Zhou, Shi Feng, Abulhair Saparov, and Ziyu Yao. A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models.arXiv preprint arXiv:2407.02646, 2024

    arXiv 2024

  64. [64]

    Open Problems in Mechanistic Interpretability .arXiv preprint arXiv:2501.16496, 2025

    Lee Sharkey, Bilal Chughtai, Joshua Batson, Jack Lindsey, Jeff Wu, Lucius Bushnaq, Nicholas Goldowsky-Dill, Stefan Heimersheim, Alejandro Ortega, Joseph Bloom, et al. Open Problems in Mechanistic Interpretability .arXiv preprint arXiv:2501.16496, 2025

    Pith/arXiv arXiv 2025

  65. [65]

    Exploring Mechanistic Interpretability in Large Language Models: Challenges, Approaches, and Insights

    Sandeep Reddy Gantla. Exploring Mechanistic Interpretability in Large Language Models: Challenges, Approaches, and Insights. In2025 International Conference on Data Science, Agents & Artificial Intelligence (ICDSAAI), pages 1–8. IEEE, 2025

    2025

  66. [66]

    Explainable AI for Medical Data: Current Methods, Limitations, and Future Directions.ACM Computing Surveys, 57(6):1–46, 2025

    Md Imran Hossain, Ghada Zamzmi, Peter R Mouton, Md Sirajus Salekin, Yu Sun, and Dmitry Goldgof. Explainable AI for Medical Data: Current Methods, Limitations, and Future Directions.ACM Computing Surveys, 57(6):1–46, 2025. 26

    2025

  67. [67]

    Explainable Artificial Intelligence for Autonomous Driving: A Comprehensive Overview and Field Guide for Future Research Directions.IEEE Access, 12:101603–101625, 2024

    Shahin Atakishiyev, Mohammad Salameh, Hengshuai Yao, and Randy Goebel. Explainable Artificial Intelligence for Autonomous Driving: A Comprehensive Overview and Field Guide for Future Research Directions.IEEE Access, 12:101603–101625, 2024

    2024

  68. [68]

    Explainable AI: the new 42? InInternational Cross-Domain Conference for Machine Learning and Knowledge Extraction, pages 295–303

    Randy Goebel, Ajay Chander, Katharina Holzinger, Freddy Lecue, Zeynep Akata, Simone Stumpf, Peter Kieseberg, and Andreas Holzinger. Explainable AI: the new 42? InInternational Cross-Domain Conference for Machine Learning and Knowledge Extraction, pages 295–303. Springer, 2018

    2018

  69. [69]

    A historical perspective of explainable Artificial Intelligence.Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 11(1):e1391, 2021

    Roberto Confalonieri, Ludovik Coba, Benedikt Wagner, and Tarek R Besold. A historical perspective of explainable Artificial Intelligence.Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 11(1):e1391, 2021

    2021

  70. [70]

    d’Avila Garcez and Luís C

    Artur S. d’Avila Garcez and Luís C. Lamb. Neurosymbolic AI: the 3rd wave.Artificial Intelligence Review, 56(11):12387–12406, 2023

    2023

  71. [71]

    Addison-Wesley Longman Publishing Co., Inc., 1984

    Bruce G Buchanan and Edward H Shortliffe.Rule based expert systems: the mycin experi- ments of the stanford heuristic programming project (the Addison-Wesley series in artificial intelligence). Addison-Wesley Longman Publishing Co., Inc., 1984

    1984

  72. [72]

    Regulation (EU) 2016/679 of the European Parliament and of the Council.Regulation (EU), 679(2016):10–13, 2016

    Protection Regulation. Regulation (EU) 2016/679 of the European Parliament and of the Council.Regulation (EU), 679(2016):10–13, 2016

    2016

  73. [73]

    A Survey on Evaluation of Large Language Models.ACM Transactions on Intelligent Systems and Technology, 15(3):1–45, 2024

    Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. A Survey on Evaluation of Large Language Models.ACM Transactions on Intelligent Systems and Technology, 15(3):1–45, 2024

    2024

  74. [74]

    A Review on Large Language Models: Architectures, Applications, Taxonomies, Open Issues and Challenges.IEEE Access, 2024

    Mohaimenul Azam Khan Raiaan, Md Saddam Hossain Mukta, Kaniz Fatema, Nur Mohammad Fahad, Sadman Sakib, Most Marufatul Jannat Mim, Jubaer Ahmad, Mohammed Eunus Ali, and Sami Azam. A Review on Large Language Models: Architectures, Applications, Taxonomies, Open Issues and Challenges.IEEE Access, 2024

    2024

  75. [75]

    Position: TrustLLM: Trustworthiness in Large Language Models

    Yue Huang, Lichao Sun, Haoran Wang, Siyuan Wu, Qihui Zhang, Yuan Li, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, et al. Position: TrustLLM: Trustworthiness in Large Language Models. InInternational Conference on Machine Learning, pages 20166–20270, 2024

    2024

  76. [76]

    Navigating the Risks: A Review of Safety Issues in Large Language Models

    Haiyang Wang, Yihao Li, Yue Wang, Pan Liu, and Pengxiao Li. Navigating the Risks: A Review of Safety Issues in Large Language Models. In2024 IEEE 24th International Conference on Software Quality, Reliability, and Security Companion (QRS-C), pages 74–83. IEEE, 2024

    2024

  77. [77]

    Why Should I Trust You?

    Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. "Why Should I Trust You?": Explaining the Predictions of Any Classifier. InProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1135–1144, 2016

    2016

  78. [78]

    Can Large Language Models Explain Themselves? A Study of LLM-Generated Self- Explanations.arXiv preprint arXiv:2310.11207, 2023

    Shiyuan Huang, Siddarth Mamidanna, Shreedhar Jangam, Yilun Zhou, and Leilani H Gilpin. Can Large Language Models Explain Themselves? A Study of LLM-Generated Self- Explanations.arXiv preprint arXiv:2310.11207, 2023

    Pith/arXiv arXiv 2023

  79. [79]

    Quantifying uncertainty in natural language explanations of large language models

    Sree Harsha Tanneru, Chirag Agarwal, and Himabindu Lakkaraju. Quantifying uncertainty in natural language explanations of large language models. InInternational Conference on Artificial Intelligence and Statistics, pages 1072–1080. PMLR, 2024

    2024

  80. [80]

    Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. Advances in Neural Information Processing Systems, 35:24824–24837, 2022

    2022

Showing first 80 references.