Pith. sign in

REVIEW 3 major objections 5 minor 46 references

Attribution methods can't localize agent failures; trace-based rubrics can.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 03:45 UTC pith:F6DEGUPW

load-bearing objection A useful conceptual framing and a clean static baseline, but the paper's central claim about attribution methods failing on agentic trajectories is not directly tested — the bridge experiment only applies SHAP to a processed rubric-feature surrogate. the 3 major comments →

arxiv 2602.06841 v4 pith:F6DEGUPW submitted 2026-02-06 cs.AI

From Features to Actions: Explainability in Traditional and Agentic AI Systems

Sindhuja Chaduvula , Jessee Ho , Kina Kim , Aravind Narayanan , Ahmed Y. Radwan , Mahshid Alinoori , Muskan Garg , Dhanesh Ramachandram
show 1 more author
Shaina Raza
This is my paper
classification cs.AI
keywords Explainable AIAgentic AIlarge language modelstrajectory-level explainabilitytool-use agentsattribution methodstrace-based rubricsstate tracking
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that explainability for AI must shift from explaining single predictions to explaining multi-step behavior. It contrasts feature-attribution methods (like SHAP and LIME) with trace-based rubric evaluations on both a static classification task and two tool-using agent benchmarks. The empirical claim is that attributions produce stable rankings for static models but cannot localize why a specific agent run failed, while trace-grounded rubrics identify per-run violations such as state-tracking inconsistency. The paper reports that state-tracking inconsistency is 2.7 times more prevalent in failed runs and cuts success probability by 49%, and it proposes a unified way to package explanations with evidence and verification.

Core claim

The central discovery is an empirical contrast: attribution methods yield stable feature rankings (Spearman ρ = 0.86) for static classifiers, but when applied to agent trajectories they only recover aggregate correlates and fail to say which constraint was violated in a particular failed run. Trace-based rubric evaluation, using an LLM judge that sees only the execution trace, flags violated behavioral constraints (intent alignment, plan adherence, tool correctness, tool choice, state consistency, error recovery) and per-run these flags localize the breakdown. The headline finding is that state-tracking inconsistency is the dominant failure mode in the airline task, 2.7x more common in faile

What carries the argument

The key instrument is the 'Minimal Explanation Packet' (MEP), a bundle that couples an explanation artifact with linked execution context and verification signals. For agentic systems the MEP is built around the trajectory (state, action, observation sequence), and verification comes from rubric-based violation flags computed trace-only by an LLM judge. The paper also defines two quantitative diagnostics: failure-mode prevalence (difference/ratio of violation flags in failed vs successful runs) and reliability correlates (difference/ratio of success probabilities with vs without a flag). These transform raw traces into per-run, actionable diagnostics.

Load-bearing premise

The rubric labels produced by the LLM judge are accurate and the execution traces capture all relevant state; if either fails, the reported prevalence and reliability numbers could be artifacts.

What would settle it

Conduct a human-expert annotation study on a sample of execution traces from these benchmarks, having experts independently label the same rubric categories as the LLM judge. If agreement is low (e.g., Cohen's kappa below 0.5) or if expert labels yield different prevalence ratios (e.g., state-tracking no longer 2.7x more common in failures), the central diagnostic claim is undermined.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Agentic XAI evaluation should be trajectory-level, not prediction-level; attribution methods should be demoted to aggregate summary tools.
  • Trace-based rubrics can serve as auditable failure-localization signals for tool-using agents, supporting debugging and oversight.
  • State-tracking consistency is a key early-warning signal; monitoring it over runs may predict failures before they become irrecoverable.
  • The same rubric framework extends across different agent tasks, showing that failure modes differ by environment (slow drift vs hard blockers).

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If rubric labels are reliable, state-tracking inconsistency could be used as a runtime alarm in deployed agents, triggering intervention before failure becomes irrecoverable.
  • The bridging experiment suggests that attribution methods can be repurposed if trajectories are first compressed into a behaviorally grounded feature space, but the compression itself requires a trace-based rubric; trace-based diagnosis is upstream of useful attribution.
  • A testable extension is applying the same rubric approach to embodied or multi-agent settings, which may reveal analogous failure modes such as state desynchronization between agents.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper argues that explainability for agentic AI systems requires a paradigm shift from feature-level attribution methods (SHAP, LIME) to trajectory-level, trace-grounded diagnostics. It proposes a formal static-vs-agentic distinction, a taxonomy of explanation targets, and a Minimal Explanation Packet (MEP) framework. Empirically, it reports stable SHAP/LIME feature rankings on a static job-posting classifier (Spearman ρ = 0.86), and then analyzes LLM-agent execution traces from TAU-bench Airline and AssistantBench using Docent rubric labels from a single GPT-5 judge. It reports that state-tracking violations are 2.7× more prevalent in failed TAU-bench runs and reduce success probability by 49%. A bridging experiment applies SHAP to a logistic regression trained on Docent rubric features, concluding that attribution methods capture only aggregate importance while trace-based rubrics provide per-run failure localization. The paper concludes that XAI for agentic systems should adopt trajectory-level explainability.

Significance. If the central negative claim were established, the paper would provide a useful conceptual correction: feature-attribution methods designed for static classifiers do not automatically transfer to multi-step agentic behavior. The taxonomy and MEP framework are reasonable organizing devices, and the release of rubric prompts/templates and code is a practical contribution. The static stability result is clean as far as it goes, and the failure-mode hypotheses for TAU-bench (state drift) versus AssistantBench (decisive tool-choice errors) are plausible and testable. However, the headline comparison between attribution and trace-based diagnosis is currently under-supported: the bridge experiment tests SHAP only on an already-abstracted rubric-feature surrogate, not on raw trajectories, and the trace-based localization claims rely on a single LLM judge with no external validation. The paper explicitly acknowledges some of these limitations in §6.3, but the admission does not close the gap between what the abstract claims and what the experiments demonstrate.

major comments (3)
  1. [§5.3, Table 10] The load-bearing negative claim in the Abstract and §1 — that attribution methods 'cannot be applied reliably to diagnose execution-level failures in agentic trajectories' — is not directly tested. In §5.3, trajectories are first converted into binary Docent rubric features, a logistic regression is trained on those features, and SHAP is applied to that surrogate. This only shows that SHAP on a trace-grounded, low-dimensional abstraction yields aggregate importance. It does not test whether SHAP or LIME applied to raw trajectories (tool calls, arguments, observations, state sequences) fails at per-run failure localization. The row 'Per-run failure localization: Limited (indirect) vs Explicit (direct)' in Table 10 is asserted, not demonstrated. The paper should either add an experiment that applies attribution methods to raw trajectory inputs and measures per-run localization accuracy, or
  2. [§3.3, §6.3, Tables 7–8] The positive claim that trace-grounded rubric evaluation 'consistently localizes behaviour breakdowns' rests entirely on labels produced by one GPT-5 judge through Docent. Section 6.3 concedes that rubric labels introduce subjectivity and depend on complete traces. The 2.7× prevalence ratio and the 49% success reduction are conditional probabilities computed from these labels. If the judge mislabels state violations or if traces omit state updates, these headline numbers could be artifacts. The manuscript needs a validation subset with human annotation or adjudication, inter-judge agreement statistics, and a sensitivity analysis of the main claims. Moreover, the rubric flags themselves are the only localization measure, so the claim that they 'localize' failures is not checked against any independent ground truth about where the failure occurred.
  3. [§5.2–5.3, Tables 7–8] The agentic experiments are based on small samples (N=50 and N=33), and no confidence intervals or significance tests are reported. Several statistics in Tables 7 and 8 involve zero cells (e.g., Ratio=∞, RR=0.00), which are uninformative without interval estimates. The headline claims about state tracking inconsistency being 2.7× more prevalent and reducing success by 49% should be accompanied by bootstrap confidence intervals or Bayesian interval estimates. Without uncertainty quantification, the cross-benchmark contrast could be driven by sparse counts, especially in AssistantBench where only 2 of 33 runs succeed.
minor comments (5)
  1. [§5.1, Table 5] The statement 'attribution methods achieve stable feature rankings in static settings' is stronger than the evidence: only one static dataset and two models are tested, and the high ρ=0.86 comes from TF–IDF + Logistic Regression. Please scope the claim to the settings studied.
  2. [§5.2, after Table 8] The interpretation that Tool Correctness 'captures minor, recoverable issues rather than fatal errors' is post-hoc and not supported by trace examples or error analysis. Either provide illustrative trajectories or soften the causal-sounding interpretation.
  3. [§5.3, Figure 4] The beeswarm plot shows SHAP values but the accompanying text says violations 'tend to push predictions toward failure.' Since the model is logistic regression, reporting coefficients would make the direction claim transparent and verifiable.
  4. [§3.4] The Minimal Explanation Packet (MEP) is introduced as a framework component but is not empirically evaluated. As written, it reads as a proposal; consider labeling it as such or adding a small user study to validate its usefulness.
  5. [References and text] Minor typos and formatting issues: reference [20] has 'NACCL' instead of 'NAACL'; the abstract contains a corrupted URL/word ('gl⌢beProject Page'); Table 6 reports inference cost but the paper never discusses it.

Circularity Check

1 steps flagged

Bridge experiment partially circular: attribution is only tested on Docent rubric flags, so its failure to localize is built into the input representation, not demonstrated on trajectories.

specific steps
  1. self definitional [Section 5.3 and Table 10]
    "Each trajectory is first labelled using Docent rubrics (Section 3.3) and then encoded as a compact binary feature vector, where each dimension indicates whether a behavioural constraint is satisfied or violated. Using these rubric features, we train a logistic regression model to predict task success vs. failure... We compute SHAP values... However, these attributions remain correlative: they explain which rubric features drive the surrogate model’s outcome predictions, not what caused a specific run to fail. Table 10: 'Per-run failure localization: Limited (indirect) vs Explicit (direct)'."

    The comparison feeds attribution only the binary rubric-feature vector produced by Docent — the same per-run violation flags that define 'explicit direct' trace-based localization. SHAP on a linear/logistic surrogate over these flags is, by construction, an aggregate importance summary; it cannot localize at a finer grain than the input features. The paper's conclusion that attribution methods cannot reliably diagnose execution-level failures in agentic trajectories is therefore enforced by the choice of input representation and by defining localization in terms of rubric flags, rather than tested by applying attribution to raw trajectories.

full rationale

The central empirical claim has two parts. The trace-based part is independently grounded: Tables 7 and 8 compute rubrics' conditional frequencies and success correlations from execution traces, and the 2.7x / 49% statistics do not reduce to the claim about attribution. The static experiment is a self-contained calibration. However, the negative claim against attribution rests on Section 5.3, where SHAP is applied only to a surrogate trained on Docent rubric features. Because the rubric flags are themselves the per-run failure localizations, SHAP's aggregate nature is preordained; the contrast in Table 10 is not a head-to-head test of attribution on trajectories. This is an experimental-design circularity rather than a formal derivation circle. The paper's self-citations ([13], [32]) are background and not load-bearing; Docent and HAL-Harness are external tools. Overall, the core static results and rubric statistics are independent, but the paradigm-shift conclusion is partially overstated by the constructed bridge experiment, warranting score 4.

Axiom & Free-Parameter Ledger

0 free parameters · 5 axioms · 1 invented entities

The paper's central claim rests on the rubric categories and the LLM judge's labels; these are adopted from prior frameworks (Docent) but not independently validated. No numerical free parameters are fit to data; the headline statistics are descriptive.

axioms (5)
  • domain assumption The six rubric categories (Intent Alignment, Plan Adherence, Tool Correctness, Tool-Choice Accuracy, State Consistency, Error Recovery) capture the relevant failure modes for tool-use agents.
    Adopted from Docent and prior agent evaluation frameworks; if a real failure mode is missing, the localization claim is incomplete. Section 3.3, Table 7.
  • domain assumption The GPT-5 judge (via Docent) labels rubric violations accurately, consistently, and without outcome leakage.
    All agentic statistics depend on these labels; the paper itself flags subjectivity in Section 6.3.
  • domain assumption HAL-Harness execution traces are complete and faithfully reflect agent state and actions.
    The paper notes this may not hold under incomplete logging (Section 6.3).
  • domain assumption Spearman rank correlation of feature attributions under perturbation is a meaningful reliability measure for static XAI.
    Standard XAI metric adopted from prior work [3,18]; used as the static-side reliability signal.
  • domain assumption The static versus agentic paradigm split is a useful organizing distinction for XAI.
    The paper's framing; if the distinction is not meaningful, the comparison loses force. Section 3.1.
invented entities (1)
  • Minimal Explanation Packet (MEP) no independent evidence
    purpose: A packaging unit that bundles explanation artifact, linked evidence, and verification signals for both static and agentic explainability.
    Conceptual framework with no falsifiable handle outside the paper's own evaluations; proposed as a standard for explanation artifacts.

pith-pipeline@v1.3.0-alltime-deepseek · 15383 in / 13510 out tokens · 116081 ms · 2026-08-03T03:45:43.113261+00:00 · methodology

0 comments
read the original abstract

Over the last decade, Explainable AI has primarily focused on interpreting individual model predictions, producing post-hoc explanations that relate inputs to outputs under a fixed decision structure. Recent advances in large language models (LLMs) have enabled agentic AI systems whose behaviour unfolds over multi-step trajectories. In these settings, success and failure are determined by sequences of decisions rather than a single output. It remains unclear how explanation approaches designed for static predictions translate to agentic settings where behaviour emerges over time. In this work, we bridge this gap by comparing attribution-based explanations with trace-based diagnostics across both settings. Our results show that while attribution methods achieve stable feature rankings in static settings (Spearman \r{ho} = 0.86), they cannot be applied reliably to diagnose execution-level failures in agentic trajectories. In contrast, trace-grounded rubric evaluation for agentic settings consistently localizes behaviour breakdowns and reveals that state tracking inconsistency is 2.7x more prevalent in failed runs and reduces success probability by 49%. These findings motivate a shift towards trajectory-level explainability for evaluating and diagnosing autonomous AI behaviour in agentic systems. Code: https://github.com/VectorInstitute/unified-xai-evaluation-framework Project page: https://vectorinstitute.github.io/unified-xai-evaluation-framework

Figures

Figures reproduced from arXiv: 2602.06841 by Ahmed Y. Radwan, Aravind Narayanan, Dhanesh Ramachandram, Jessee Ho, Kina Kim, Mahshid Alinoori, Muskan Garg, Shaina Raza, Sindhuja Chaduvula.

Figure 1
Figure 1. Figure 1: Comparison of Minimal Explanation Packet (MEP) structure across static [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Agent execution loop with explicit state–action–observation semantics. [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Comparison of local (LIME) (a), and global (SHAP) (b) interpretability. [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: SHAP summary (beeswarm) plot for rubric-level features. Each point is [PITH_FULL_IMAGE:figures/full_fig_p018_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

46 extracted references · 1 canonical work pages

  1. [1]

    arXiv preprint arXiv:2005.00928 (2020)

    Abnar, S., Zuidema, W.: Quantifying attention flow in transformers. arXiv preprint arXiv:2005.00928 (2020)

    Pith/arXiv arXiv 2005

  2. [2]

    arXiv preprint arXiv:1610.01644 (2016)

    Alain, G., Bengio, Y.: Understanding intermediate layers using linear classifier probes. arXiv preprint arXiv:1610.01644 (2016)

    Pith/arXiv arXiv 2016

  3. [3]

    Advances in Neural Information Processing Systems (2018)

    Alvarez-Melis, D., Jaakkola, T.: Towards robust interpretability with self- explaining neural networks. Advances in Neural Information Processing Systems (2018)

    2018

  4. [4]

    Arize AI: Phoenix: Agent trajectory evaluation for llm systems (2024),https: //arize.com/ai-agents/agent-evaluation/

    2024

  5. [5]

    arXiv preprint arXiv:2507.08443 (2025)

    Balanos, G., Chasanis, E., Skianis, K., Pitoura, E.: Kgrag-ex: Explainable retrieval- augmented generation with knowledge graph-based perturbations. arXiv preprint arXiv:2507.08443 (2025)

    Pith/arXiv arXiv 2025

  6. [6]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Chefer, H., Gur, S., Wolf, L.: Transformer interpretability beyond attention visu- alization. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 782–791 (2021).https://doi.org/10.1109/CVPR46437. 2021.00084

    arXiv 2021

  7. [7]

    arXiv preprint arXiv:2307.08678 (2023)

    Chen, Y., Li, R., Tan, C., Wang, Y.T.: Do models explain themselves? counterfactual simulatability of natural language explanations. arXiv preprint arXiv:2307.08678 (2023)

    Pith/arXiv arXiv 2023

  8. [8]

    ConfidentAI:Deepeval:Aframeworkforevaluatingllm-basedapplications.https: //deepeval.com/guides/guides-ai-agent-evaluation-metrics(2023)

    2023

  9. [9]

    Advances in Neural Information Processing Systems arXiv:2304.14997 (2023)

    Conmy, A., Mavor-Parker, A., Lynch, A., Heimersheim, S., Garriga-Alonso, A.: Towards automated circuit discovery for mechanistic interpretability. Advances in Neural Information Processing Systems arXiv:2304.14997 (2023)

    Pith/arXiv arXiv 2023

  10. [10]

    arXiv preprint arXiv:2309.08600 (2023)

    Cunningham, H., Ewart, A., Riggs, L., Huben, R., Sharkey, L.: Sparse autoen- coders find highly interpretable features in language models. arXiv preprint arXiv:2309.08600 (2023)

    Pith/arXiv arXiv 2023

  11. [11]

    arXiv preprint arXiv:1702.08608 (2017)

    Doshi-Velez,F.,Kim,B.:Towardsarigorousscienceofinterpretablemachinelearn- ing. arXiv preprint arXiv:1702.08608 (2017)

    Pith/arXiv arXiv 2017

  12. [12]

    In: Proceedings of the XAI Conference (2024).https: //doi.org/10.1007/978-3-031-63800-8_9

    Engelhart, M., Wörner, S., Althoff, M.: Evaluation of shap for explainable re- inforcement learning. In: Proceedings of the XAI Conference (2024).https: //doi.org/10.1007/978-3-031-63800-8_9

    work page doi:10.1007/978-3-031-63800-8_9 2024

  13. [13]

    Farooq, A., Raza, S., Karim, M.N., Iqbal, H., Vasilakos, A.V., Emmanouilidis, C.: Evaluating and regulating agentic ai: A study of benchmarks, metrics, and regulation (2025)

    2025

  14. [14]

    The Annals of Statistics (2001),https://doi.org/10.1214/aos/1013203451

    Friedman, J.H.: Greedy function approximation: A gradient boosting machine. The Annals of Statistics (2001),https://doi.org/10.1214/aos/1013203451

    arXiv 2001

  15. [15]

    ACM computing surveys (CSUR) arXiv:1802.01933 (2018)

    Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., Pedreschi, D.: A survey of methods for explaining black box models. ACM computing surveys (CSUR) arXiv:1802.01933 (2018)

    Pith/arXiv arXiv 2018

  16. [16]

    Gunning, D.: Explainable artificial intelligence (xai). Tech. rep., Defense Advanced Research Projects Agency (DARPA) (2017),https://www.darpa.mil/program/ explainable-artificial-intelligence, dARPA XAI Program Overview Explainability in Traditional and Agentic AI Systems 23

    2017

  17. [17]

    Hewitt, J., Manning, C.D.: A structural probe for finding syntax in word repre- sentations. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume1(LongandShortPapers).pp.4129–4138(2019),https://aclanthology. org/N19-1419/

    2019

  18. [18]

    Advances in Neural Information Processing Systems arXiv:1806.10758 (2019)

    Hooker, S., Erhan, D., Kindermans, P.J., Kim, B.: A benchmark for interpretabil- ity methods in deep neural networks. Advances in Neural Information Processing Systems arXiv:1806.10758 (2019)

    Pith/arXiv arXiv 2019

  19. [19]

    Springer Cham (2025),https: //doi.org/10.1007/978-3-031-90026-6

    Huang, K.: Agentic AI: Theories and Practices. Springer Cham (2025),https: //doi.org/10.1007/978-3-031-90026-6

    doi:10.1007/978-3-031-90026-6 2025

  20. [20]

    NACCL arXiv:2201.08164 (2019)

    Jain, S., Wallace, B.C.: Attention is not explanation. NACCL arXiv:2201.08164 (2019)

    Pith/arXiv arXiv 2019

  21. [21]

    Kapoor, S., Stroebl, B., Others: Holistic agent leaderboard: The missing infras- tructure for ai agent evaluation (2025)

    2025

  22. [22]

    arXiv preprint arXiv:2506.12152 (2025)

    Kim, B., Hewitt, J., Nanda, N., Fiedel, N., Tafjord, O.: Because we have llms, we can and should pursue agentic interpretability. arXiv preprint arXiv:2506.12152 (2025)

    Pith/arXiv arXiv 2025

  23. [23]

    In: Proceedings of the 35th International Conference on Machine Learning (ICML) arXiv.1711.11279 (2018)

    Kim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viégas, F.B., Sayres, R.: Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In: Proceedings of the 35th International Conference on Machine Learning (ICML) arXiv.1711.11279 (2018)

    Pith/arXiv arXiv 2018

  24. [24]

    arXiv preprint arXiv:2308.07198 (2023)

    Krause, J., Montavon, G., Samek, W., Müller, K.R.: Explaining black-box models through counterfactuals. arXiv preprint arXiv:2308.07198 (2023)

    Pith/arXiv arXiv 2023

  25. [25]

    Lundberg, S.M., Lee, S.I.: A unified approach to interpreting model predictions (2017), advances in Neural Information Processing Systems arXiv 1705.07874

    Pith/arXiv arXiv 2017

  26. [26]

    Meng, K., Huang, V., Steinhardt, J., Schwettmann, S.: Introducing docent.https: //transluce.org/introducing-docent(March 2025)

    2025

  27. [27]

    Distillhttps://doi.org/10.23915/distill.00024 (2020)

    Olah, C., Cammarata, N., Schubert, L., Goh, G., Petrov, M., Carter, S.: Zoom in: An introduction to circuits. Distillhttps://doi.org/10.23915/distill.00024 (2020)

    doi:10.23915/distill.00024 2020

  28. [28]

    OpenAI: Gpt-4.1 model documentation (2025),https://openai.com/index/ gpt-4-1/

    2025

  29. [29]

    OpenAI: Introducing o4-mini (2025),https://openai.com/index/ introducing-o3-and-o4-mini/

    2025

  30. [30]

    OpenAI: Openai gpt-5 system card (2025),https://openai.com/index/ gpt-5-system-card/

    2025

  31. [31]

    Advances in Neural Infor- mation Processing Systems37, 135783–135818 (2024)

    Parekh, J., Khayatan, P., Shukor, M., Newson, A., Cord, M.: A concept-based explainability framework for large multimodal models. Advances in Neural Infor- mation Processing Systems37, 135783–135818 (2024)

    2024

  32. [32]

    In Proceedings of the AAAI 2023 Spring Symposium: Responsible Medical AI, Design, and Operationalization

    Raza, S., Pour, P.O., Bashir, S.R.: Fairness in Machine Learning meets with Equity in Healthcare. In Proceedings of the AAAI 2023 Spring Symposium: Responsible Medical AI, Design, and Operationalization. arXiv:2305.07041 (2023)

    Pith/arXiv arXiv 2023

  33. [33]

    Radiology: artificial intelligence2(3), e190043 (2020).https://doi.org/10.1148/ryai.2020190043

    Reyes, M., Meier, R., Pereira, S., Silva, C.A., Dahlweid, F.M., Tengg-Kobligk, H.v., Summers, R.M., Wiest, R.: On the interpretability of artificial intelligence in radiology: challenges and opportunities. Radiology: artificial intelligence2(3), e190043 (2020).https://doi.org/10.1148/ryai.2020190043

    doi:10.1148/ryai.2020190043 2020

  34. [34]

    why should i trust you?

    Ribeiro, M.T., Singh, S., Guestrin, C.: "why should i trust you?": Explaining the predictions of any classifier (2016), published in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2016) arXiv:1602.04938 24 S. Chaduvula et al

    Pith/arXiv arXiv 2016

  35. [35]

    Selvaraju, R.R., Das, A., Vedantam, R., Cogswell, M., Parikh, D., Batra, D.: Grad- cam: Why did you say that? arXiv preprint arXiv:1611.07450 (2016)

    Pith/arXiv arXiv 2016

  36. [36]

    In: NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following arXiv:2303.11366 (2023)

    Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., Yao, S.: Reflex- ion: An autonomous agent with dynamic memory and self-reflection. In: NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following arXiv:2303.11366 (2023)

    Pith/arXiv arXiv 2023

  37. [37]

    arXiv preprint arXiv:1312.6034 (2013)

    Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional net- works: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034 (2013)

    Pith/arXiv arXiv 2013

  38. [38]

    In: Proceedings of the 34th International Conference on Machine Learning arXiv:1703.01365 (2017)

    Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep networks. In: Proceedings of the 34th International Conference on Machine Learning arXiv:1703.01365 (2017)

    Pith/arXiv arXiv 2017

  39. [39]

    VibrantLabsAI: Agentic or tool use.https://docs.ragas.io/en/stable/ concepts/metrics/available_metrics/agents/(2025)

    2025

  40. [40]

    arXiv preprint arXiv:1904.02679 (2019)

    Vig, J.: A multiscale visualization of attention in the transformer model. arXiv preprint arXiv:1904.02679 (2019)

    Pith/arXiv arXiv 1904

  41. [41]

    Harvard Journal of Law & Technology arXiv:1711.00399 (2017)

    Wachter, S., Mittelstadt, B., Russell, C.: Counterfactual explanations without opening the black box: Automated decisions and the gdpr. Harvard Journal of Law & Technology arXiv:1711.00399 (2017)

    Pith/arXiv arXiv 2017

  42. [42]

    arXiv preprint arXiv:2211.00593 (2022)

    Wang, K., Variengien, A., Conmy, A., Shlegeris, B., Steinhardt, J.: Interpretability in the wild: a circuit for indirect object identification in gpt-2 small. arXiv preprint arXiv:2211.00593 (2022)

    Pith/arXiv arXiv 2022

  43. [43]

    arXiv preprint arXiv:2201.11903 (2022)

    Wei, J., Wang, X., Schuurmans, D., Bosma, M., et al.: Chain-of-thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903 (2022)

    Pith/arXiv arXiv 2022

  44. [44]

    arXiv preprint arXiv:2406.12045 (2024)

    Yao, S., Shinn, N., Razavi, P., Narasimhan, K.:τ-bench: A benchmark for tool- agent-user interaction in real-world domains. arXiv preprint arXiv:2406.12045 (2024)

    Pith/arXiv arXiv 2024

  45. [45]

    arXiv preprint arXiv:2210.03629 (2022)

    Yao, S., et al.: React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629 (2022)

    Pith/arXiv arXiv 2022

  46. [46]

    Yoran, O., Amouyal, S.J., Malaviya, C., Bogin, B., Press, O., Berant, J.: Assistant- bench: Can web agents solve realistic and time-consuming tasks? In: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. pp. 8938–8968 (2024)

    2024