Pith. sign in

REVIEW 4 major objections 6 minor 92 references

Integrating Counterfactual Simulations with Language Models for Explaining Multi-Agent Behaviour

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that letting an LLM interrogate a driving simulator with counterfactual 'what if' and 'remove' prompts produces action explanations that score at least 7.7% higher on perceived correctness than a context-only baseline.

desk verdict AXIS is a genuinely new and well-engineered framework for multi-agent action explanations; the headline numbers, however, rest on an LLM judge that has not been calibrated to human ratings. read the letter →

arxiv 2505.17801 v2 pith:XBZFVATK submitted 2025-05-23 cs.AI

Bálint Gyevnár , Christopher G. Lucas , Stefano V. Albrecht , Shay B. Cohen This is my paper · ORCID
classification cs.AI
keywords multi-agentsystemsexplainableAIcounterfactualexplanationslargelanguagemodelsautonomousdrivingLLM-as-a-judgeactioneffectsizemodel
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that action explanations for multi-agent systems can be generated by a loop in which a large language model interrogates an environment simulator with counterfactual prompts, such as removing an agent or asking what would happen if it changed lanes, and then synthesises the simulated outcomes into natural-language explanations. The proposed method, AXIS, grounds this loop in the counterfactual effect size model: a cause is worth citing when intervening on it changes whether the queried outcome occurs under otherwise similar conditions. On ten autonomous-driving scenarios across five LLMs, AXIS improved perceived explanation correctness by at least 7.7% relative to a context-only baseline for every model, improved goal-prediction accuracy by up to 23% for four models, and kept action-prediction accuracy comparable. The paper also contributes an evaluation template that combines qualitative analysis, robustness checks, feature-contribution analysis, subjective preference, and objective goal and action prediction using a separate LLM as judge.

What carries the argument

The load-bearing mechanism is the interrogation-synthesis loop, mediated by four typed prompts—add, remove, whatif, what—that translate between the LLM's natural-language proposals and the simulator's low-level state and action space. Actions are wrapped into options so the LLM can reason over named, temporal abstractions rather than raw acceleration and steering vectors, and observations, options, and reward components are verbalised into text the model can use. The counterfactual effect size model supplies the selection criterion: an intervention that changes whether the queried outcome happens identifies a cause worth citing.

What would settle it

Run the same ten scenarios and user prompts with human participants, scoring preference, perceived correctness, and goal and next-action prediction, and compare those scores with the external LLM judge's scores. If the two disagree substantially, or if AXIS no longer beats the context-only baseline under human scoring, the claimed improvements are an artifact of the judge rather than of the explanations.

Watch

Extended reading notes

Core claim

AXIS operationalises the counterfactual effect size model computationally by replacing a one-shot LLM explanation with a multi-round interrogation-synthesis loop. The LLM proposes interventions in the form of high-level options, such as remove(1) or whatif(0, change-lane, 40), the simulator rolls the world forward under that intervention, and the LLM synthesises the new observations and rewards into a partial explanation before proposing the next intervention. After up to ten rounds, the full prompt history is assembled into a final explanation. The paper reports that this process makes explanations select the correct causes in rational driving scenarios, that remove and whatif are the interrogation types models rely on most, and that across five LLMs AXIS beats the context-only baseline on perceived correctness by between 7.67% and 17.83%, while improving goal prediction for four of five models.

Load-bearing premise

The reported gains assume the external LLM judge scores explanations the way human users would, since no human subjects were run and the paper itself flags this as a limitation.

Editorial extensions

If this is right

  • AXIS can be applied to any multi-agent system that has a simulator or a learned forward model, because it never inspects the agents' internal policies.
  • Because models typically emit a stop signal before exhausting the ten-round budget (average maximum round 5.03), useful explanations can be produced with a modest number of simulations.
  • The feature-contribution analysis gives a per-model recipe for which context channels to include, so prompt length can be reduced without sacrificing correctness.
  • The paper's own qualitative results indicate that gains are concentrated in rational, fully observed scenarios, while irrational agents and occluded vehicles remain failure modes.
  • The five-part evaluation template offers a reusable protocol for future work on explainable sequential decision-making, combining qualitative, robustness, preference, correctness, and actionability measures.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A cheap check on the paper's central claim would be a small human study on a subset of the ten scenarios; the 7.7% headline stands or falls on how well the LLM judge's scores track human judgments.
  • Because the failures on irrational and occluded scenarios come from LLMs assuming rational, fully observed behaviour, a testable extension is priming the model with explicit hypotheses about rule violations or hidden vehicles before it proposes interventions.
  • The same interrogation-synthesis loop could transfer to discrete multi-agent domains such as warehouse routing or level-based foraging, though the paper notes that scaling to many agents is untested.
  • Replacing the hand-built simulator with a learned world model would let AXIS explain black-box multi-agent policies in settings where no simulator exists.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces AXIS, a framework that uses an LLM to interrogate a simulator with counterfactual prompts (add, remove, whatif, what) over multiple rounds and synthesize explanations for multi-agent action queries. It formalizes action explanation in partially observable stochastic games, proposes the AXIS algorithm, and evaluates it in autonomous-driving scenarios with five LLM generators and an external LLM judge. The headline results claim that AXIS improves perceived explanation correctness by at least 7.7% across all models and goal-prediction accuracy by 23% for four of five LLMs, with comparable action-prediction accuracy, and the implementation is open-sourced.

Significance. If the quantitative claims held, AXIS would be a practical, model-agnostic addition to explainable multi-agent systems, and its evaluation methodology would be a useful template combining robustness, qualitative correctness, subjective preference, and downstream goal/action prediction. The formalization of the explanation problem (Section 2.1), the use of options to bridge LLM and simulator representations, and the release of code are concrete strengths. The central risk is that the headline perceptual claims rest entirely on an unvalidated LLM judge for metrics that are inherently about human judgments; as reported, the evidence supports claims about LLM-judged quality, not human-perceived quality.

major comments (4)
  1. [Section 4, 'Metrics' and 'LLM-as-a-Judge'; Section 6.2, limitation (1)] The three starred measures (subjective preference, perceived correctness, goal/action prediction) are introduced as user-study metrics, yet all reported scores are produced by Claude 3.5, and the paper provides no validation that this judge tracks human ratings. Section 6.2 explicitly acknowledges that LLM assessment, 'especially on subjective preference, is biased.' This is load-bearing because the abstract's headline statements are about 'perceived explanation correctness' and 'goal prediction accuracy.' Without a calibration study (e.g., correlation or agreement between Claude 3.5 and human raters on a subset of explanations), the quantitative claims should be reworded as LLM-judged quality, and the current wording overstates the evidence.
  2. [Section 4, 'Metrics', best-round selection] The aggregate results in Table 3 compare AXIS with ModelOnly using 'best-round explanations' selected by the highest geometric mean of preference and correctness across interrogation-synthesis rounds. ModelOnly has no corresponding selection step. Because Figure 5 shows non-monotonic performance across rounds, this oracle-style selection can only favor AXIS and may inflate the reported improvements. The paper should either use a fixed protocol (e.g., first round, final round, or average over rounds) or apply the same selection procedure to the baseline.
  3. [Section 5.2, Table 3] The headline mean differences are reported without inferential statistics. For example, GPT-4.1 correctness is 3.00 ± 0.27 vs. 3.23 ± 0.34, and several preference and action-accuracy comparisons show overlapping standard errors. Claims such as 'at least 7.7%' and '23%' are therefore not yet established as statistically reliable; paired tests or confidence intervals over the ten scenarios and prompts would be needed to support the comparative claims.
  4. [Section 4, 'Shapley Analysis of Context Features'] The context features used in the full experiments are selected using Shapley values computed on scenarios #3, #7, and #8, and the same scenario types (rational, irrational, occlusion) appear in the main evaluation set. This creates a selection-leakage path: the feature set is tuned on part of the evaluation data, which can overstate model-dependent improvements. The paper should report results separately for the three tuning scenarios or demonstrate that the conclusions are unchanged on the remaining seven scenarios.
minor comments (6)
  1. [Section 4 and Tables 2-3] The model list in Section 4 includes o1-2024-12-17, but Tables 2 and 3 report results for o4-mini; the naming inconsistency should be resolved.
  2. [References] References [77] and [78] appear to be the same Shapley value reference with different page ranges; they should be merged into a single entry.
  3. [Figure 2] The caption says 'Scenario #3 (left; rational)', but the left panel is labeled 'Scenario 2'; the scenario numbering in the figure should be aligned with the text and caption.
  4. [Section 5.2] The phrase 'from 23.73% to 46%' is inconsistent with the goal-accuracy deltas in Table 3 (+.14 to +.23); please clarify what quantity this range refers to.
  5. [Section 3, first paragraph] The phrase 'An important question with using an LLM' should be 'An important question when using an LLM'.
  6. [Algorithm 1] The initial call to VerbaliseContext at line 2 passes an empty reward, and the first synthesis phase also has no reward signal; the paper should explain why the initial context omits rewards and how the LLM handles their absence in early rounds.

Circularity Check

1 steps flagged · score 6.0 of 10

Headline correctness gain is partially forced by selecting the best round on the same perceived-correctness metric being reported; the AXIS algorithm itself is not definitionally circular.

  1. fitted input called prediction [Section 4 (Evaluation Methodology, best-round selection) and Section 5.2 (Table 3 caption)]
    "When presenting aggregate results, we select best-round explanations which have the highest geometric mean of preference and perceived correctness scores across interrogation-synthesis rounds. ... Comparison of baselines and AXIS on metrics aggregated across scenarios and prompts, with AXIS using best-round explanations."

    Table 3's AXIS correctness and preference numbers are, by definition, maxima over the same LLM-judged scores across rounds: 'best-round' means the round with the highest geometric mean of preference and perceived correctness, and the table then reports the correctness/preference of that selected round as AXIS's score. The ModelOnly and NoExp baselines receive no analogous selection over candidate outputs. The reported 'at least 7.7%' perceived-correctness improvement is therefore partly an optimised upper envelope over intermediate explanations rather than a property of the single final explanation returned by Algorithm 1. The selection variable and the reported outcome are the same quantity, so the headline comparison is partially forced by construction.

full rationale

The AXIS framework itself is not definitionally circular: it uses an external simulator and an LLM to propose and synthesize counterfactuals, and CESM [73] is cited as motivating background rather than as a derivational input. The LLM judge is held out from the generator models, so the evaluation is not self-scoring by construction. The clearest circular step is the best-round selection: Section 4 defines the reported 'best-round' explanation as the one with the highest preference/perceived-correctness scores, and Section 5.2 then reports those same scores as AXIS's headline improvement. This statistically forces part of the perceived-correctness gain, since AXIS gets to take a maximum over round-level variability while baselines do not. The Shapley-based feature selection on perceived correctness over a scenario subset that also contributes to the aggregate correctness results, and the absence of human calibration for the LLM judge, are additional validity risks, but they are not needed for the circularity finding. Self-citations to [3], [36], and [73] include author overlap but are used as simulator, metric, and psychological background respectively, not as proof of the central empirical claim. Score 6 reflects that one headline quantitative claim partially reduces to a selection rule by construction, while the method's design and most actionability results retain independent content.

Assumptions & free parameters 2 free parameters · 4 assumptions · 1 invented entities

The method itself is a framework rather than a mathematical derivation, so the ledger is dominated by domain assumptions about the simulator, the LLM's intervention ability, the validity of LLM-as-a-judge, and the psychological model. The only data-dependent choices are the per-model feature sets selected via Shapley analysis and the hand-set maximum-round parameter. No new unobserved entities are introduced.

free parameters (2)
  • Per-model context feature set (Shapley-selected) = Larger models: options + states + complex linguistic content; smaller model: options + actions + concise linguistic…
    Selected using Shapley value analysis on perceived correctness for scenarios #3, #7, #8 (Section 4, Shapley Analysis). The tuned configurations are then used in the full evaluation, so the reported scores are not fully out-of-sample with respect to this choice.
  • Maximum interrogation rounds N_max = 10
    Set by hand in Section 4 (Models). The paper analyzes sensitivity to this via intermediate-round scores in Figure 5, with models on average terminating after 5.03 rounds, so this is a loose upper bound rather than a tightly fitted parameter.
assumptions (4)
  • domain assumption A simulator bT exists that approximates the transition function T and the joint action policy pi.
    Section 3 states: 'the main assumption of AXIS is the existence of a simulator bT which approximates the state transition function T and the joint action policy pi.' The paper uses IGP2 for autonomous driving, so this is satisfied in the evaluated domain but not guaranteed for arbitrary MAS.
  • domain assumption LLMs can propose meaningful interventions and synthesize useful counterfactual explanations.
    Section 3 and the qualitative results in Section 5.1. The paper notes that models below 7B parameters fail to propose meaningful interventions and that irrational/occluded scenarios are poorly handled, so this assumption holds only partially and is a known limitation.
  • domain assumption An external LLM judge (Claude 3.5) is a valid proxy for human evaluation of explanation quality and actionability.
    Section 4, 'LLM-as-a-Judge' paragraph, and acknowledged as biased in Section 6.2, limitation (1). The central quantitative claims about perceived correctness and goal/action prediction rely directly on this assumption.
  • domain assumption The counterfactual effect size model (CESM) describes how humans select causes for explanations.
    Section 3 invokes CESM from Quillien and Lucas (2023), with author overlap, to justify the interrogation design. This is a background psychological model rather than a theorem proved in this paper.
invented entities (1)
  • Interrogation prompt primitives (add, remove, whatif, what) independent evidence
    purpose: Provide a structured communication interface between the LLM and the simulator so the LLM can propose counterfactual interventions.
    These primitives are fully specified in Table 1 and implemented in the open-source code, so they are concrete artifacts of the system rather than unobserved postulated entities. They do not introduce a hidden causal mechanism.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Integrating Counterfactual Simulations with Language Models for Explaining Multi-Agent Behaviour." pith.science (2026). https://pith.science/paper/XBZFVATK

@misc{pith2026250517801,
  author       = {Pith},
  title        = {Pith review of: Integrating Counterfactual Simulations with Language Models for Explaining Multi-Agent Behaviour},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XBZFVATK}},
  note         = {Machine review of arXiv:2505.17801}
}
read the original abstract

Autonomous multi-agent systems (MAS) are useful for automating complex tasks but raise trust concerns due to risks such as miscoordination or goal misalignment. Explainability is vital for users' trust calibration, but explainable MAS face challenges due to complex environments, the human factor, and non-standardised evaluation. Leveraging the counterfactual effect size model and LLMs, we propose Agentic eXplanations via Interrogative Simulation (AXIS). AXIS generates human-centred action explanations for multi-agent policies by having an LLM interrogate an environment simulator using prompts like 'whatif' and 'remove' to observe and synthesise counterfactual information over multiple rounds. We evaluate AXIS on autonomous driving across ten scenarios for five LLMs with a comprehensive methodology combining robustness, subjective preference, correctness, and goal/action prediction with an external LLM as evaluator. Compared to baselines, AXIS improves perceived explanation correctness by at least 7.7% across all models and goal prediction accuracy by 23% for four models, with comparable action prediction accuracy, achieving the highest scores overall. Our code is open-sourced at https://github.com/gyevnarb/axis.

Figures

Figures reproduced from arXiv: 2505.17801 by the authors.

Figure 1
Figure 1. When a user asks a question about an agent’s actions, AXIS retrieves the current context, which the LLM uses [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Example scenarios with the queried vehicle shown in blue, other vehicles in orange/white, and occluded vehicles in red. Scenario #3 (left; rational): blue sees orange come to a stop, indicating orange’s intent to turn left, so blue decides to turn right instead of waiting longer to yield. Scenario #8 (right; occlusion): blue sees orange on a priority lane coming to a stop, inferring that red must be behind the build… view at source ↗
Figure 3
Figure 3. Shapley values calculated from perceived correct [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Evolution of preference/correctness (top), and [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 4
Figure 4. Figure 4: The proportion of queries across all scenarios and [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

92 extracted references · 38 canonical work pages

  1. [37]

    Lucas, Shay B

    Balint Gyevnar, Cheng Wang, Christopher G. Lucas, Shay B. Cohen, and Ste- fano V. Albrecht. 2024. Causal Explanations for Sequential Decision-Making in Multi-Agent Systems. InProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems(Auckland, New Zealand)(AAMAS ’24). International Foundation for Autonomous Agents and Mul...

    work page 2024

  2. [1]

    David Abel, Dilip Arumugam, Lucas Lehnert, and Michael Littman. 2018. State Abstractions for Lifelong Reinforcement Learning. InProceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 80), Jennifer Dy and Andreas Krause (Eds.). PMLR, 10–19

    2018

  3. [2]

    Sahar Admoni, Omer Ben-Porat, and Ofra Amir. 2025. SySLLM: Generating Syn- thesized Policy Summaries for Reinforcement Learning Agents Using Large Lan- guage Models. https://doi.org/10.48550/arXiv.2503.10509 arXiv:2503.10509 [cs]

    work page doi:10.48550/arxiv.2503.10509 2025

  4. [3]

    Albrecht, Cillian Brewitt, John Wilhelm, Balint Gyevnar, Francisco Eiras, Mihai Dobre, and Subramanian Ramamoorthy

    Stefano V. Albrecht, Cillian Brewitt, John Wilhelm, Balint Gyevnar, Francisco Eiras, Mihai Dobre, and Subramanian Ramamoorthy. 2021. Interpretable Goal- based Prediction and Planning for Autonomous Driving. InIEEE International Conference on Robotics and Automation (ICRA). 1043–1049. https://doi.org/10. 1109/ICRA48506.2021.9560849

    arXiv 2021

  5. [4]

    Albrecht, Filippos Christianos, and Lukas Schäfer

    Stefano V. Albrecht, Filippos Christianos, and Lukas Schäfer. 2024.Multi-Agent Reinforcement Learning: Foundations and Modern Approaches. MIT Press. https: //www.marl-book.com

    2024

  6. [5]

    Stefano V Albrecht and Subramanian Ramamoorthy. 2016. Exploiting causality for selective belief filtering in dynamic Bayesian networks.Journal of Artificial Intelligence Research55 (2016), 1135–1178

    2016

  7. [6]

    Albrecht and Peter Stone

    Stefano V. Albrecht and Peter Stone. 2018. Autonomous agents modelling other agents: A comprehensive survey and open problems.Artificial Intelligence258 (2018), 66–95. https://doi.org/10.1016/j.artint.2018.01.002

    doi:10.1016/j.artint.2018.01.002 2018

  8. [7]

    Ziyan An, Hendrik Baier, Abhishek Dubey, Ayan Mukhopadhyay, and Meiyi Ma

Show all 92 references
  1. [8]

    Richardson, Austin C

    Jacy Reese Anthis, Ryan Liu, Sean M. Richardson, Austin C. Kozlowski, Bernard Koch, James Evans, Erik Brynjolfsson, and Michael Bernstein. 2025. LLM Social Simulations Are a Promising Research Method. arXiv 2504.02234

    2025 arXiv

  2. [9]

    Anthropic. 2025. Agentic Misalignment. https://www.anthropic.com/research/ agentic-misalignment

    2025

  3. [10]

    Anthropic. 2025. Claude 3.5 Haiku. https://www.anthropic.com/claude Large language model

    2025

  4. [11]

    Ahsaas Bajaj, Pavitra Dangati, Kalpesh Krishna, Pradhiksha Ashok Kumar, Rheeya Uppaal, Bradford Windsor, Eliot Brenner, Dominic Dotterrer, Rajarshi Das, and Andrew McCallum. 2021. Long Document Summarization in a Low Resource Setting Using Pretrained Language Models. InProceed...

    2021

  5. [12]

    Tom Bewley and Jonathan Lawry. 2021. TripleTree: A Versatile Interpretable Representation of Black Box Agents and Their Environments.Proceedings of the AAAI Conference on Artificial Intelligence35, 13 (May 2021), 11415–11422. https://doi.org/10.1609/aaai.v35i13.17360

    2021 doi

  6. [13]

    Kayla Boggess, Sarit Kraus, and Lu Feng. 2022. Toward Policy Explanations for Multi-Agent Reinforcement Learning. InThirty-First International Joint Confer- ence on Artificial Intelligence, Vol. 1. 109–115. https://doi.org/10.24963/ijcai.2022/ 16

    2022 doi

  7. [14]

    Kayla Boggess, Sarit Kraus, and Lu Feng. 2023. Explainable Multi-Agent Rein- forcement Learning for Temporal Queries. InProceedings of the Thirty-Second In- ternational Joint Conference on Artificial Intelligence (IJCAI ’23). Macao, P.R.China, 55–63. https://doi.org/10.24963/i...

    2023 doi

  8. [15]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey W...

    2020

  9. [16]

    Ruth M.J. Byrne. 2023. Good explanations in explainable artificial intelligence (XAI): evidence from human explanatory reasoning. InProceedings of the Thirty- Second International Joint Conference on Artificial Intelligence (IJCAI ’23). Inter- national Joint Conferences on Art...

    2023 doi

  10. [17]

    Ruth M. J. Byrne. 2005.The rational imagination: how people create alternatives to reality. The MIT Press, Cambridge, MA. https://doi.org/10.7551/mitpress/5756. 001.0001

    2005 doi

  11. [18]

    Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al . 2024. A survey on evaluation of large language models.ACM transactions on intelligent systems and technology15, 3 (2024), 1–45

    2024

  12. [19]

    Filippos Christianos, Lukas Schäfer, and Stefano V Albrecht. 2020. Shared Ex- perience Actor-Critic for Multi-Agent Reinforcement Learning. InAdvances in Neural Information Processing Systems (NeurIPS)

    2020

  13. [20]

    Council of Europe. 2024. The Framework Convention on Artificial Intelli- gence. https://www.coe.int/en/web/artificial-intelligence/the-framework- convention-on-artificial-intelligence Accessed: 2025-04-20

    2024

  14. [21]

    Richard Dazeley, Peter Vamplew, and Francisco Cruz. 2023. Explainable Re- inforcement Learning for Broad-XAI: A Conceptual Framework and Survey. Neural Computing and Applications35, 23 (Aug. 2023), 16893–16916. https: //doi.org/10.1007/s00521-023-08423-1

    2023 doi

  15. [22]

    Richard Dazeley, Peter Vamplew, Cameron Foale, Charlotte Young, Sunil Aryal, and Francisco Cruz. 2021. Levels of Explainable Artificial Intelligence for Human- Aligned Conversational Explanations.Artificial Intelligence299 (2021), 103525. https://doi.org/10.1016/j.artint.2021.103525

    2021

  16. [23]

    Francesco Bombassei De Bona, Gabriele Dominici, Tim Miller, Marc Langheinrich, and Martin Gjoreski. 2024. Evaluating Explanations Through LLMs: Beyond Traditional User Studies. arXiv:2410.17781 [cs.AI] https://arxiv.org/abs/2410. 17781

    2024 arXiv

  17. [24]

    DeepSeek. 2025. DeepSeek-R1. https://deepseekv3.org/deepseek-r1

    2025

  18. [25]

    DeepSeek. 2025. DeepSeek-V3 Technical Report. https://deepseekv3.org/paper

    2025

  19. [26]

    Stephanie Droop and Neil R Bramley. 2023. Extending counterfactual reasoning models to capture unconstrained social explanations

    2023

  20. [27]

    Dzindolet, Scott A

    Mary T. Dzindolet, Scott A. Peterson, Regina A. Pomranky, Linda G. Pierce, and Hall P. Beck. 2003. The Role of Trust in Automation Reliance.International Journal of Human-Computer Studies58, 6 (June 2003), 697–718. https://doi.org/ 10.1016/S1071-5819(03)00038-7

    2003 doi

  21. [28]

    Vera Liao, Larry Chan, I-Hsiang Lee, Michael Muller, and Mark O Riedl

    Upol Ehsan, Samir Passi, Q. Vera Liao, Larry Chan, I-Hsiang Lee, Michael Muller, and Mark O Riedl. 2024. The Who in XAI: How AI Background Shapes Perceptions of AI Explanations. InProceedings of the CHI Conference on Human Factors in Computing Systems (CHI ’24). Association fo...

    2024

  22. [29]

    Upol Ehsan and Mark O. Riedl. 2020. Human-Centered Explainable AI: To- wards a Reflective Sociotechnical Approach. InHCI International 2020 - Late Breaking Papers: Multimodality and Intelligence (Lecture Notes in Computer Science), Constantine Stephanidis, Masaaki Kurosu, Helm...

    2020 doi

  23. [30]

    Jasmina Gajcin and Ivana Dusparic. 2024. Redefining Counterfactual Explanations for Reinforcement Learning: Overview, Challenges and Opportunities.ACM Comput. Surv.56, 9 (April 2024), 219:1–219:33. https://doi.org/10.1145/3648472

    2024 doi

  24. [31]

    Jasmina Gajcin, Jovan Jeromela, and Ivana Dusparic. 2024. Semifactual Expla- nations for Reinforcement Learning. InProceedings of the 12th International Conference on Human-Agent Interaction (HAI ’24). Association for Computing Ma- chinery, New York, NY, USA, 167–175. https://...

    2024

  25. [32]

    Daniel Geschke, Jan Lorenz, and Peter Holtz. 2018. The triple-filter bubble: Using agent-based modelling to test a meta-theoretical framework for the emergence of filter bubbles and echo chambers.British Journal of Social Psychology58, 1 (2018), 129–149. https://doi.org/10.111...

    2018 doi

  26. [33]

    Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, Saizhuo Wang, Kun Zhang, Yuanzhuo Wang, Wen Gao, Lionel Ni, and Jian Guo. 2025. A Survey on LLM-as- a-Judge. https://doi.org/10.48550/arXiv.2411.15594

    2025 doi arXiv

  27. [34]

    Cohen, Neil R

    Balint Gyevnar, Stephanie Droop, Tadeg Quillien, Shay B. Cohen, Neil R. Bramley, Christopher G. Lucas, and Stefano V. Albrecht. 2025. People Attribute Purpose to Autonomous Vehicles When Explaining Their Behavior: Insights from Cognitive Science for Explainable AI. InProceedin...

    2025

  28. [35]

    Balint Gyevnar, Nick Ferguson, and Burkhard Schafer. 2023. Bridging the Transparency Gap: What Can Explainable AI Learn From the AI Act?. In 26th European Conference on Artificial Intelligence. IOS Press, 964–971. https: //doi.org/10.3233/FAIA230367

    2023 doi

  29. [36]

    Balint Gyevnar and Mark Towers. 2025. Objective Metrics for Human-Subjects Evaluation in Explainable Reinforcement Learning. In2025 Multi-Disciplinary Conference on Reinforcement Learning and Decision Making(Dublin, Ireland)

    2025

  30. [38]

    2016.Actual Causality

    Joseph Y Halpern. 2016.Actual Causality. MIT Press, Cambridge, MA

    2016

  31. [39]

    Lewis Hammond, Alan Chan, Jesse Clifton, Jason Hoelscher-Obermaier, Akbir Khan, Euan McLean, Chandler Smith, Wolfram Barfuss, Jakob Foerster, Tomáš Gavenčiak, The Anh Han, Edward Hughes, Vojtěch Kovařík, Jan Kulveit, Joel Z. Leibo, Caspar Oesterheld, Christian Schroeder de Wit...

    2025 doi arXiv

  32. [40]

    Hanna, Arrasy Rahman, Elliot Fosong, Francisco Eiras, Mihai Dobre, John Redford, Subramanian Ramamoorthy, and Stefano V

    Josiah P. Hanna, Arrasy Rahman, Elliot Fosong, Francisco Eiras, Mihai Dobre, John Redford, Subramanian Ramamoorthy, and Stefano V. Albrecht. 2021. Inter- pretable Goal Recognition in the Presence of Occluded Factors for Autonomous Vehicles. InIEEE/RSJ International Conference ...

    2021

  33. [41]

    Alexandre Heuillet, Fabien Couthouis, and Natalia Díaz-Rodríguez. 2022. Collec- tive eXplainable AI: Explaining Cooperative Strategies and Agent Contribution in Multiagent Reinforcement Learning With Shapley Values.Comp. Intell. Mag. 17, 1 (Feb. 2022), 59–71. https://doi.org/1...

    2022

  34. [42]

    Hoffman, Shane T

    Robert R. Hoffman, Shane T. Mueller, Gary Klein, and Jordan Litman. 2023. Measures for Explainable AI: Explanation Goodness, User Satisfaction, Mental Models, Curiosity, Trust, and Human-AI Performance.Frontiers in Computer Science5 (2023)

    2023

  35. [43]

    Tobias Huber, Katharina Weitz, Elisabeth André, and Ofra Amir. 2021. Local and Global Explanations of Agent Behavior: Integrating Strategy Summaries with Saliency Maps.Artificial Intelligence301 (Dec. 2021), 103571. https://doi.org/10. 1016/j.artint.2021.103571

    2021

  36. [44]

    IASEAI. 2025. IASEAI Call to Action for Lawmakers, Academics, and the Public Ahead of AI Summit in Paris. https://www.iaseai.org/conference/statement Accessed: 2025-04-20

    2025

  37. [45]

    Nidhi Kalra and Susan M. Paddock. 2016. Driving to Safety: How Many Miles of Driving Would It Take to Demonstrate Autonomous Vehicle Relia- bility?Transportation Research Part A: Policy and Practice94 (2016), 182–193. https://doi.org/10.1016/j.tra.2016.09.010

    2016 doi

  38. [46]

    Margot E Kaminski. 2021. The right to explanation, explained.Berkeley Technology Law Journal34 (2021), 189–218. https://doi.org/10.15779/Z38TD9N83H

    2021 doi

  39. [47]

    Help Me Help the AI

    Sunnie S. Y. Kim, Elizabeth Anne Watkins, Olga Russakovsky, Ruth Fong, and Andrés Monroy-Hernández. 2023. "Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interaction. InProceedings of the 2023 CHI Conference on Human Factors in Computing Systems (C...

    2023

  40. [48]

    Kyle, Mehrdad Samadi, and Tugkan Tuzun

    Andrei Kirilenko, Albert S. Kyle, Mehrdad Samadi, and Tugkan Tuzun. 2017. The Flash Crash: High-Frequency Trading in an Electronic Market.Journal of Finance 72, 3 (2017), 967–998. https://doi.org/10.1111/jofi.12498

    2017 doi

  41. [49]

    Albrecht

    Anton Kuznietsov, Balint Gyevnar, Cheng Wang, Steven Peters, and Stefano V. Albrecht. 2024. Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic Review.IEEE Transactions on Intelligent Transportation Systems25, 12 (Dec. 2024), 19342–19364. https://doi.org/1...

    2024

  42. [50]

    Philippe Laban, Hiroaki Hayashi, Yingbo Zhou, and Jennifer Neville. 2025. LLMs Get Lost In Multi-Turn Conversation. arXiv:arXiv 2505.06120 https://arxiv.org/ abs/2505.06120

    2025 arXiv

  43. [51]

    Philippe Laban, Hiroaki Hayashi, Yingbo Zhou, and Jennifer Neville. 2025. LLMs Get Lost In Multi-Turn Conversation. https://doi.org/10.48550/arXiv.2505.06120 arXiv:2505.06120 [cs]

    2025 doi arXiv

  44. [52]

    Bruno Lepri, Nuria Oliver, Emmanuel Letouze, Alex Pentland, and Patrick Vinck

  45. [53]

    Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang

    Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the Middle: How Language Models Use Long Contexts.Transactions of the Association for Computational Linguistics 12 (2024), 157–173. https://doi.org/10.116...

    2024 doi

  46. [54]

    Qinghua Liu, Alan Chung, Csaba Szepesvári, and Chi Jin. 2022. When is partially observable reinforcement learning not scary?. InConference on Learning Theory. PMLR, 5175–5220

    2022

  47. [55]

    Llama Team, AI at Meta. 2024. The Llama 3 Herd of Models. arXiv 2407.21783 [cs]

    2024 arXiv

  48. [56]

    Helena Löfström, Karl Hammar, and Ulf Johansson. 2022. A Meta Survey of Quality Evaluation Criteria in Explanation Methods. InIntelligent Information Systems, Jochen De Weerdt and Artem Polyvyanyy (Eds.). Springer International Publishing, 55–63

    2022

  49. [57]

    Tania Lombrozo. 2006. The structure and function of explanations.Trends in cognitive sciences10, 10 (2006), 464–470. https://doi.org/10.1016/j.tics.2006.08.004

    2006 doi

  50. [58]

    Christopher G Lucas and Charles Kemp. 2015. An improved probabilistic account of counterfactual reasoning.Psychological review122, 4 (2015), 700

    2015

  51. [59]

    Prashan Madumal, Tim Miller, Liz Sonenberg, and Frank Vetere. 2020. Ex- plainable Reinforcement Learning through a Causal Lens.Proceedings of the AAAI Conference on Artificial Intelligence34, 03 (April 2020), 2493–2500. https://doi.org/10.1609/aaai.v34i03.5631

    2020 doi

  52. [60]

    Malle and Joshua Knobe

    Bertram F. Malle and Joshua Knobe. 1997. Which Behaviors Do People Explain? A Basic Actor–Observer Asymmetry.Journal of Personality and Social Psychology 72, 2 (1997), 288–304. https://doi.org/10.1037/0022-3514.72.2.288

    1997 doi

  53. [61]

    Peta Masters and Sebastian Sardina. 2019. Goal recognition for rational and irrational agents. InProceedings of the 18th international conference on autonomous agents and multiagent systems. 440–448

    2019

  54. [62]

    Stephanie Milani, Nicholay Topin, Manuela Veloso, and Fei Fang. 2024. Ex- plainable Reinforcement Learning: A Survey and Comparative Review.Comput. Surveys56, 7 (April 2024), 168:1–168:36. https://doi.org/10.1145/3616864

    2024 doi

  55. [63]

    Tim Miller. 2019. Explanation in Artificial Intelligence: Insights from the Social Sciences.Artificial Intelligence267 (Feb. 2019), 1–38. https://doi.org/10.1016/j. artint.2018.07.007

    2019 doi

  56. [64]

    Tim Miller. 2023. Explainable AI Is Dead, Long Live Explainable AI! Hypothesis- driven Decision Support Using Evaluative AI. InProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’23). Association for Computing Machinery, New York, NY, U...

    2023

  57. [65]

    Sina Mohseni, Niloofar Zarei, and Eric D. Ragan. 2021. A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems.ACM Transactions on Interactive Intelligent Systems11, 3-4 (Aug. 2021), 24:1–24:45. https://doi.org/10.1145/3387166

    2021 doi

  58. [66]

    Mahsan Nourani, Samia Kabir, Sina Mohseni, and Eric D. Ragan. 2019. The Effects of Meaningful and Meaningless Explanations on Trust and Perceived System Accuracy in Intelligent Systems.Proceedings of the AAAI Conference on Human Computation and Crowdsourcing7 (Oct. 2019), 97–105

    2019

  59. [67]

    O’Laughlin and Bertram F

    Matthew J. O’Laughlin and Bertram F. Malle. 2002. How People Explain Actions Performed by Groups and Individuals.Journal of Personality and Social Psychology 82, 1 (2002), 33–48. https://doi.org/10.1037//0022-3514.82.1.33

    2002 doi

  60. [68]

    Olson, Roli Khanna, Lawrence Neal, Fuxin Li, and Weng-Keen Wong

    Matthew L. Olson, Roli Khanna, Lawrence Neal, Fuxin Li, and Weng-Keen Wong

  61. [69]

    OpenAI. 2024. OpenAI o1 System Card. https://openai.com/index/openai-o1- system-card/

    2024

  62. [70]

    OpenAI. 2025. Introducing GPT-4.1 in the API. https://openai.com/index/gpt-4-1/

    2025

  63. [71]

    Albrecht

    Georgios Papoudakis, Filippos Christianos, Lukas Schäfer, and Stefano V. Albrecht

  64. [72]

    Erika Puiutta and Eric MSP Veith. 2020. Explainable reinforcement learning: A survey. InInternational cross-domain conference for machine learning and knowl- edge extraction. Springer, 77–95

    2020

  65. [73]

    Tadeg Quillien and Christopher G. Lucas. 2023. Counterfactuals and the Logic of Causal Selection.Psychological Review131, 5 (2023), 1208—-1234. https: //doi.org/10.31234/osf.io/ts76y

    2023 doi

  66. [74]

    Anand S Rao, Michael P Georgeff, et al. 1995. BDI agents: From theory to practice.. InICMAS, Vol. 95. 312–319

    1995

  67. [75]

    InProceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks (NeurIPS)

    Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks. InProceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks (NeurIPS). http://arxiv.org/abs/2006.07869

    2006 arXiv

  68. [76]

    Wilko Schwarting, Javier Alonso-Mora, and Daniela Rus. 2018. Planning and Decision-Making for Autonomous Vehicles.Annual Review of Control, Robotics, and Autonomous Systems1, 1 (2018), 187–210. https://doi.org/10.1146/annurev- control-060117-105157

    2018 doi

  69. [77]

    L. Shapley. 1953. A Value for N-Person Games. InClassics in Game Theory. Princeton University Press, Chapter Classics in Game Theory, 69–79

    1953

  70. [78]

    L. Shapley. 2020. A Value for N-Person Games. InClassics in Game Theory. Princeton University Press, Chapter Classics in Game Theory, 69–79

    2020

  71. [79]

    Cynthia Rudin. 2019. Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead.Nature Machine Intelligence1, 5 (May 2019), 206–215. https://doi.org/10.1038/s42256-019-0048-x

    2019 doi

  72. [80]

    Mark Towers, Yali Du, Christopher Freeman, and Timothy J. Norman. 2024. Explaining an Agent’s Future Beliefs Through Temporally Decomposing Future Reward Estimators. InECAI 2024. IOS Press, 2790–2797. https://doi.org/10.3233/ FAIA240814

    2024

  73. [81]

    Christian Unkelbach and Rainer Greifeneder. 2013. A general model of fluency effects in judgment and decision making. InThe experience of thinking. Psychology Press, 11–32

    2013

  74. [82]

    George A. Vouros. 2022. Explainable Deep Reinforcement Learning: State of the Art and Challenges.ACM Comput. Surv.55, 5, Article 92 (2022), 39 pages. https://doi.org/10.1145/3527448

    2022 doi

  75. [83]

    Sutton, Doina Precup, and Satinder Singh

    Richard S. Sutton, Doina Precup, and Satinder Singh. 1999. Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learn- ing.Artificial Intelligence112, 1 (1999), 181–211. https://doi.org/10.1016/S0004- 3702(99)00052-1

    1999 doi

  76. [84]

    Qingyue Wang, Yanhe Fu, Yanan Cao, Shuai Wang, Zhiliang Tian, and Liang Ding. 2025. Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Models.Neurocomputing639 (July 2025), 130193. https: //doi.org/10.1016/j.neucom.2025.130193

    2025

  77. [85]

    Michael Winikoff. 2017. Debugging Agent Programs with Why? Questions. In Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems (AAMAS ’17). International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, 251–259

    2017

  78. [86]

    Michael Winikoff, Galina Sidorenko, Virginia Dignum, and Frank Dignum. 2021. Why bad coffee? Explaining BDI agent behaviour with valuings.Artificial Intelli- gence300 (2021), 103554

    2021

  79. [87]

    Sandra Wachter, Brent Mittelstadt, and Chris Russell. 2017. Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harvard Journal of Law & Technology31, 2 (2017), 841–887. https://doi.org/10. 2139/ssrn.3063289

    2017

  80. [88]

    Hashimoto

    Tianyi Zhang, Faisal Ladhak, Esin Durmus, Percy Liang, Kathleen McKeown, and Tatsunori B. Hashimoto. 2024. Benchmarking Large Language Models for News Summarization.Transactions of the Association for Computational Linguistics12 (2024), 39–57. https://doi.org/10.1162/tacl_a_00632

    2024 doi

  81. [91]

    Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the Effect of Accuracy on Trust in Machine Learning Models. InProceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19). Association for Computing Machinery, New York, NY, U...

    2019

  82. [2018]

    Philosophy & Technology31, 4 (2018), 611–627

    Fair, transparent, and accountable algorithmic decision-making processes. Philosophy & Technology31, 4 (2018), 611–627. https://doi.org/10.1007/s13347- 017-0279-x

    2018 doi

  83. [2021]

    https://doi.org/10.1016/j.artint.2021.103455

    Counterfactual State Explanations for Reinforcement Learning Agents via Generative Deep Learning.Artificial Intelligence295 (June 2021), 103455. https://doi.org/10.1016/j.artint.2021.103455

    2021

  84. [2024]

    InProceedings of the 27th European Conference on Artificial Intelligence (ECAI)

    Enabling MCTS Explainability for Sequential Planning Through Compu- tation Tree Logic. InProceedings of the 27th European Conference on Artificial Intelligence (ECAI)

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.