Pith. sign in

REVIEW 2 major objections 6 minor 1 cited by

Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safety

T0 review · 2 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This survey claims to be the first unified map of LLM interpretation for safety, organized by workflow stage.

desk verdict A genuinely useful map of interpretation-for-safety, with a faithfulness caveat that should be addressed in revision rather than treated as a fatal flaw. read the letter →

arxiv 2506.05451 v1 pith:FQ5LFIOM submitted 2025-06-05 cs.SE cs.AIcs.CL

Seongmin Lee , Aeree Cho , Grace C. Kim , ShengYun Peng , Mansi Phute , Duen Horng Chau This is my paper
classification cs.SEcs.AIcs.CL
keywords LLMsafetyinterpretabilitysurveytaxonomytrainingdataattributionsparseautoencoderschain-of-thoughtjailbreak
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Large language models can behave unsafely, and interpretation methods promise to show why; this paper argues that the two research lines have matured enough to be systematically connected, and that no survey has done so. It claims the connection is not one-directional: interpretation can reveal causes of unsafe outputs, those causes can inform concrete safety enhancements, and tools can operationalize both. To make the bridge navigable, it organizes nearly 70 works into a taxonomy based on where interpretation operates in the LLM workflow: training process, input prompts, inference-time internals, and self-reasoning during generation. A sympathetic reader would care because the paper turns a scattered literature into a map with named stages, enhancement strategies, and tool types, plus a list of open problems.

What carries the argument

The central object is the workflow-stage taxonomy: interpretation methods are binned by where they touch the model — training process, input prompts, inference internals, and generation for self-reasoning. This taxonomy does the load-bearing work of the survey: it anchors the cross-references to six safety enhancement strategies and four tool types, and it exposes which combinations are populated and which are empty. The accompanying intersection tables are the concrete mechanism, letting a reader see at a glance which works connect which interpretation stage, safety concern, enhancement, and tool.

What would settle it

Run intervention tests on the flagged causes: take the tokens, latent directions, neurons, or reasoning steps that an interpretation method marks as responsible for an unsafe output, remove or steer them, and check whether the unsafe behavior disappears. If a substantial share of such interventions leave the behavior unchanged, or if an LLM's self-explanation of a harmful response is shown to be post-hoc rationalization rather than the mechanism, the central premise that interpretation improves safety loses its footing.

Watch

Extended reading notes

Core claim

The paper's central claim is that the intersection of LLM interpretation and safety is now a coherent research area, and it provides the first unified framework for navigating it. The framework connects four groups of safety-focused interpretation methods — training-data attribution, input-token analysis, inference-time internal-state analysis (probing, perturbation, and latent-vector deciphering), and self-explanatory generation — to six safety enhancement strategies they inform, from steering latent vectors and modulating neurons to verifying outputs and reasoning before answering. It then links both to four types of practical tools: training-data attribution visualizations, token visualizations, latent-vector visualizations, and neuron visualizations. The survey organizes its nearly 70 selected works in a table that marks which safety concerns (hallucination, jailbreak/harmfulness, bias, privacy leakage) each work addresses, and it closes with open challenges, including defense against interpretation-based attacks and reliable evaluation of interpretation. Appendix tables extend the map to safety-oriented interpretation methods not yet leveraged for safety enhancement.

Load-bearing premise

The whole interpretation-to-safety pipeline assumes that the surveyed interpretation methods are faithful, meaning they identify what actually causes the model's unsafe behavior rather than what merely correlates with it, and the paper itself notes that self-generated and perturbation-based explanations can be unreliable.

Editorial extensions

If this is right

  • A researcher can locate unexplored cells: the appendix explicitly lists safety-oriented interpretation methods not yet used for safety enhancement, marking candidate directions.
  • A practitioner can pick a tool by workflow stage: token, latent-vector, neuron, or training-data attribution visualizations, with libraries easing implementation.
  • Self-reasoning interpretation becomes a first-class category, so chain-of-thought and post-hoc explanation work is now surveyed alongside mechanistic methods.
  • The open-challenges list identifies concrete next problems: defending against interpretation-based attacks, standardizing faithfulness evaluation, and applying training attribution to safety.
  • The framework gives future surveys a shared vocabulary for reporting which safety concern, workflow stage, and tool a new method touches.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the taxonomy predicts that methods sharing a workflow stage are more likely to transfer across safety concerns; for example, token-level attribution found for jailbreaks could be tested directly on bias or privacy leakage.
  • Beyond the paper: the weakest link identified in Sections 3.4 and 6 — unreliable self-reasoning — suggests a concrete test: measure whether interventions based on an interpretation change the targeted unsafe behavior; without that, the enhancement claims are conditional.
  • Beyond the paper: if training-data attribution ever scales to proprietary LLM training data, the framework points to a new mitigation class: removing or downweighting the specific data points that cause unsafe behavior, something the survey notes is currently limited.
  • Beyond the paper: the user-centered presentation gap implies that interactive and conversational explanation interfaces remain an open design space for LLM safety.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper surveys roughly seventy works at the intersection of LLM interpretation and safety, organizing them under a unified framework that connects safety-focused interpretation methods (§3), the safety enhancement strategies they inform (§4), and the tools that operationalize those enhancements (§5). Interpretation methods are categorized by LLM workflow stage: training-process attribution, input-token attribution, inference-time internals, and self-reasoning. The survey focuses on four safety concerns (hallucination, jailbreaks/harmfulness, bias, privacy leakage), restricts attention to autoregressive Transformer-based generative LLMs, and closes with open challenges (§6), limitations (§7), and potential risks (§8). The central contribution is taxonomic: the paper claims to be the first survey bridging interpretation and safety through this stage-based framework.

Significance. If the survey's framing is accepted, it fills a real gap: prior surveys largely treat interpretation and safety separately, and the workflow-stage taxonomy is a useful organizing device for a fast-moving literature. The manuscript is candid about its own limits: §7 states that full technical details are omitted and that actionability remains an open question, and §8 explicitly warns that readers may mistakenly assume interpretation guarantees safety. This candor is a strength. The paper makes no quantitative or machine-checkable claims, so its value lies in coverage and organization rather than in new empirical results. The handful of self-citations (e.g., LLM Attributor) appear among the surveyed tools and do not drive the framework's conclusions. However, the central interpretation-to-safety link is presented largely as a co-occurrence in tables rather than as a critically evaluated relationship, which is the main risk in the manuscript's framing.

major comments (2)
  1. [§4, Table 1] The enhancement strategies in §4 are presented as being informed by the interpretation methods of §3, but the manuscript itself documents that several of those method families can be unfaithful: §3.2 states that perturbation methods can create out-of-distribution inputs that cause unfaithful interpretations, §3.4 states that self-reasoning explanations can be unreliable, and §6 warns that misleading interpretations can lead to overtrust and unsafe decisions. Because Table 1 and the accompanying taxonomy assign works to interpretation-to-enhancement intersections without any annotation of reliability or evidence strength, the framework can be read as endorsing the very link that the text elsewhere cautions against. The revision should operationalize the existing caveats, for example by adding a per-family faithfulness discussion at the start of §4 or a quality/evidence annotation in Table 1, so that the taxonomy does not implicitly assert that interpretation guarantees safety.
  2. [§2, Tables 1–3] The survey selection protocol is not reproducible. §2 says the authors 'curated nearly 70 works from top venues' but does not specify search databases, query terms, inclusion/exclusion criteria, screening steps, or the time window of coverage. For a survey whose central claim is to be the first comprehensive bridge between interpretation and safety, the absence of a transparent selection protocol weakens the reader's ability to verify coverage and assess omitted work. The revision should add a brief methodology paragraph describing the selection process, and the tables should include a legend defining what a colored cell means and how the appendix tables relate to Table 1, since the appendix captions say 'extending' but Tables 2 and 3 repeat rows from Table 1.
minor comments (6)
  1. [§7] There is a typo in the Limitations section: 'widely used an studied models' should read 'widely used and studied models.'
  2. [§4.3] The phrase 'resamping tokens' should read 'resampling tokens.'
  3. [§3.3.1] The sentence 'A simple yet powerful approaches compute mean latent vectors' has a subject-verb agreement error; it should be 'A simple yet powerful approach computes mean latent vectors.'
  4. [§5.4] The clause 'helping concept identification and SAE circuit discovery' is slightly awkward; consider 'supporting concept identification and SAE circuit discovery.'
  5. [Appendix, Tables 2–3] The appendix tables appear to duplicate several rows already present in Table 1 (e.g., Hazra et al. 2024, Zhao et al. 2024e, Lee et al. 2025a). If these tables are intended to be cumulative or extended, the captions and column headings should state this explicitly; otherwise the duplication confuses the relationship between the main table and the appendix.
  6. [Table 1] The legend for the colored cells is not included in the text version; without a legend defining 'colored cell' and the mapping of colors to safety types, techniques, and tools, the table is difficult for a reader to verify or reuse.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the survey synthesizes external literature; its taxonomy is organizational rather than derivational, and its few self-citations are illustrative, not load-bearing.

full rationale

This is a survey paper without a mathematical derivation chain, fitted parameters, or predictive model. The central contribution is a taxonomy that organizes roughly 70 external works by LLM workflow stage and connects them to safety enhancement strategies and tools. That framing is a categorization of the surveyed literature, not a result derived from its own definitions. The paper explicitly grounds its notion of interpretation in established sources (Doshi-Velez and Kim, 2017; Räuker et al., 2023) and repeatedly attributes findings to the cited primary works. The few self-citations, such as the authors' LLM Attributor tool in §5.1 and their earlier visualization work, serve as example entries in the survey tables; removing them would not change the framework or any claim. The paper also flags its own epistemic limitation rather than hiding it: §8 warns that 'readers may overly rely on certain techniques or mistakenly assume that interpretation guarantees safety,' and §3.4 notes that self-reasoning explanations 'can be unreliable.' Flagging an open problem about faithfulness is not a circular step; it is a caveat about the field being surveyed. Because there is no derivation from inputs to conclusions, no fitted value renamed as a prediction, and no load-bearing self-citation chain, the appropriate finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters or invented entities are introduced. The paper's central contribution is a taxonomy, which is an organizational scheme rather than a new physical or conceptual entity.

assumptions (3)
  • domain assumption The survey focuses only on autoregressive Transformer-based generative LLMs; other architectures are excluded.
    Stated in §2; the taxonomy and conclusions may not generalize beyond this architecture.
  • domain assumption Interpretation is defined as extracting knowledge from an LLM to explain its behaviors in human-understandable terms.
    Adopted in §2 from established definitions; shapes which methods are considered interpretation.
  • domain assumption Only four safety concerns are considered: hallucination, jailbreaks and harmfulness, bias, and privacy leakage.
    Stated in §2 and §8; other risks such as code security or over-refusal are out of scope.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safety." pith.science (2026). https://pith.science/paper/FQ5LFIOM

@misc{pith2026250605451,
  author       = {Pith},
  title        = {Pith review of: Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safety},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FQ5LFIOM}},
  note         = {Machine review of arXiv:2506.05451}
}
read the original abstract

As large language models (LLMs) see wider real-world use, understanding and mitigating their unsafe behaviors is critical. Interpretation techniques can reveal causes of unsafe outputs and guide safety, but such connections with safety are often overlooked in prior surveys. We present the first survey that bridges this gap, introducing a unified framework that connects safety-focused interpretation methods, the safety enhancements they inform, and the tools that operationalize them. Our novel taxonomy, organized by LLM workflow stages, summarizes nearly 70 works at their intersections. We conclude with open challenges and future directions. This timely survey helps researchers and practitioners navigate key advancements for safer, more interpretable LLMs.

Figures

Figures reproduced from arXiv: 2506.05451 by the authors.

Figure 1
Figure 1. Visual overview our survey’s unified framework bridging LLM interpretation and safety, summarizing the [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Step-Wise Refusal Dynamics in Autoregressive and Diffusion Language Models

    cs.LG 2026-02 conditional novelty 6.0 of 10

    Diffusion language models can revise away harmful intermediate text, and a step-wise internal refusal signal detects jailbreaks cheaply across autoregressive and diffusion models.

Reference graph

Works this paper leans on

44 extracted references · 16 canonical work pages · cited by 1 Pith paper

  1. [3]

    Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski

    Refusal in language models is mediated by a single direction.arXiv preprint arXiv:2406.11717. Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski. 2018. Linear algebraic struc- ture of word senses, with applications to polysemy. Transactions of the Association for Computational Linguistics, 6:483–495. Leila Arras, Bruno Puri, Patrick K...

    arXiv 2018

  2. [4]

    InThe Thirteenth International Con- ference on Learning Representations

    Mechanistic permutability: Match features across layers. InThe Thirteenth International Con- ference on Learning Representations. Sarah Ball, Frauke Kreuter, and Nina Panickssery. 2024. Understanding jailbreak success: A study of latent space dynamics in large language models.arXiv preprint arXiv:2406.09289. Oren Barkan, Yehonatan Elisha, Yonatan Toib, Jo...

    arXiv 2024

  3. [5]

    InThe Thirty-eighth Annual Conference on Neural Information Processing Systems

    Truth is universal: Robust detection of lies in LLMs. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems. Collin Burns, Haotian Ye, Dan Klein, and Jacob Stein- hardt. 2022. Discovering latent knowledge in lan- guage models without supervision.arXiv preprint arXiv:2212.03827. Vivien Cabannes, Charles Arnal, Wassim Bouaziz, Xingy...

    arXiv 2022

  4. [6]

    V ojtech Cahlik, Rodrigo Alves, and Pavel Kordik

    Iteration head: A mechanistic study of chain- of-thought.Advances in Neural Information Process- ing Systems, 37:109101–109122. V ojtech Cahlik, Rodrigo Alves, and Pavel Kordik

    work page

  5. [7]

    Nitay Calderon and Roi Reichart

    Reasoning-grounded natural language ex- planations for language models.arXiv preprint arXiv:2503.11248. Nitay Calderon and Roi Reichart. 2025. On behalf of the stakeholders: Trends in NLP model interpretabil- ity in the era of LLMs. InProceedings of the 2025 Conference of the Nations of the Americas Chap- ter of the Association for Computational Linguisti...

    work page arXiv 2025

  6. [8]

    Improving steering vectors by targeting sparse autoencoder features.Preprint, arXiv:2411.02193. Tyler A. Chang, Dheeraj Rajagopal, Tolga Bolukbasi, Lucas Dixon, and Ian Tenney. 2025. Scalable influ- ence and fact tracing for large language model pre- training. InThe Thirteenth International Conference on Learning Representations. Aditya Chattopadhyay, Piy...

    arXiv 2025

  7. [9]

    Jianhui Chen, Xiaozhi Wang, Zijun Yao, Yushi Bai, Lei Hou, and Juanzi Li

    Finetuning language models to emit lin- guistic expressions of uncertainty.arXiv preprint arXiv:2409.12180. Jianhui Chen, Xiaozhi Wang, Zijun Yao, Yushi Bai, Lei Hou, and Juanzi Li. 2024a. Finding safety neurons in large language models.Preprint, arXiv:2406.14144. Yida Chen, Aoyu Wu, Trevor DePodesta, Catherine Yeh, Kenneth Li, Nicholas Castillo Marin, Oa...

    arXiv 2025

  8. [11]

    In Proceedings of the 29th International Conference on Intelligent User Interfaces, IUI ’24, page 787–802, New York, NY , USA

    iscore: Visual analytics for interpreting how language models automatically score summaries. In Proceedings of the 29th International Conference on Intelligent User Interfaces, IUI ’24, page 787–802, New York, NY , USA. Association for Computing Machinery. Ian Covert, Scott Lundberg, and Su-In Lee. 2021. Ex- plaining by removing: A unified framework for m...

    arXiv 2021

Show all 44 references
  1. [12]

    InThe Eleventh International Conference on Learning Rep- resentations

    Selection-inference: Exploiting large language models for interpretable logical reasoning. InThe Eleventh International Conference on Learning Rep- resentations. David Dale, Elena V oita, Loic Barrault, and Marta R. Costa-jussà. 2023. Detecting and mitigating halluci- nations ...

    2023

  2. [13]

    InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 16124–16170, Toronto, Canada

    Analyzing transformers in embedding space. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 16124–16170, Toronto, Canada. Association for Computational Linguistics. Xander Davies, Max Nadeau, Nikhil Prakas...

    2023 arXiv

  3. [15]

    Han Guo, Nazneen Rajani, Peter Hase, Mohit Bansal, and Caiming Xiong

    Studying large language model general- ization with influence functions.arXiv preprint arXiv:2308.03296. Han Guo, Nazneen Rajani, Peter Hase, Mohit Bansal, and Caiming Xiong. 2021. FastIF: Scalable influ- ence functions for efficient model interpretation and debugging. InProce...

    2021 arXiv

  4. [17]

    InProceedings of the 37th Interna- tional Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY , USA

    How does gpt-2 compute greater-than? inter- preting mathematical abilities in a pre-trained lan- guage model. InProceedings of the 37th Interna- tional Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY , USA. Curran Associates Inc. Michael Hanna, Sand...

  5. [18]

    Shibo Hao, Yi Gu, Haotian Luo, Tianyang Liu, Xiyan Shao, Xinyuan Wang, Shuhua Xie, Haodi Ma, Adithya Samavedhi, Qiyue Gao, Zhen Wang, and Zhiting Hu

    Have faith in faithfulness: Going beyond cir- cuit overlap when finding model mechanisms.arXiv preprint arXiv:2403.17806. Shibo Hao, Yi Gu, Haotian Luo, Tianyang Liu, Xiyan Shao, Xinyuan Wang, Shuhua Xie, Haodi Ma, Adithya Samavedhi, Qiyue Gao, Zhen Wang, and Zhiting Hu. 2024....

    2024 arXiv

  6. [19]

    Zhengfu He, Xuyang Ge, Qiong Tang, Tianxiang Sun, Qinyuan Cheng, and Xipeng Qiu

    Towards llm guardrails via sparse representa- tion steering.arXiv preprint arXiv:2503.16851. Zhengfu He, Xuyang Ge, Qiong Tang, Tianxiang Sun, Qinyuan Cheng, and Xipeng Qiu. 2024d. Dictio- nary learning improves patch-free circuit discovery in mechanistic interpretability: A c...

  7. [20]

    Alon Jacovi and Yoav Goldberg

    How llms learn: Tracing internal represen- tations with sparse autoencoders.arXiv preprint arXiv:2503.06394. Alon Jacovi and Yoav Goldberg. 2020. Towards faith- fully interpretable NLP systems: How should we define and evaluate faithfulness? InProceedings of the 58th Annual Me...

    2020

  8. [21]

    InICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5

    Comt: Chain-of-medical-thought reduces hal- lucination in medical report generation. InICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. Zhuoran Jin, Pengfei Cao, Hongbang Yuan, Yubo Chen, Jiexin Xu, Huaijun Li, Xi...

    2025 arXiv

  9. [23]

    InProceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY , USA

    Post hoc explanations of language models can improve language models. InProceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY , USA. Curran Associates Inc. Bum Chul Kwon and Nandana Mihindukulasooriya

  10. [24]

    InProceedings of the 61st Annual Meeting of the Association for Compu- tational Linguistics (Volume 3: System Demonstra- tions), pages 42–50, Toronto, Canada

    Finspector: A human-centered visual in- spection tool for exploring and comparing biases among foundation models. InProceedings of the 61st Annual Meeting of the Association for Compu- tational Linguistics (Volume 3: System Demonstra- tions), pages 42–50, Toronto, Canada. Asso...

  11. [25]

    InThe Twelfth International Conference on Learning Repre- sentations

    Datainf: Efficiently estimating data influence in loRA-tuned LLMs and diffusion models. InThe Twelfth International Conference on Learning Repre- sentations. Biagio La Rosa, Graziano Blasilli, Romain Bourqui, David Auber, Giuseppe Santucci, Roberto Capo- bianco, Enrico Bertini...

    2023

  12. [26]

    InProceedings of the 61st An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11482– 11498, Toronto, Canada

    Contrastive error attribution for finetuned language models. InProceedings of the 61st An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11482– 11498, Toronto, Canada. Association for Computa- tional Linguistics. Himabindu Lakkara...

    2022 arXiv

  13. [27]

    In Proceedings of the 41st International Conference on Machine Learning, ICML’24

    A mechanistic understanding of alignment algorithms: a case study on dpo and toxicity. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org. Seongmin Lee, Zijie J Wang, Aishwarya Chakravarthy, Alec Helbling, ShengYun Peng, Mansi Phute, Due...

    2024 arXiv

  14. [28]

    Https://hdsr.mitpress.mit.edu/pub/aelql9qy

    AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap.Har- vard Data Science Review, (Special Issue 5). Https://hdsr.mitpress.mit.edu/pub/aelql9qy. Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, János Kramár,...

    2024 arXiv

  15. [29]

    Niclas Luick

    Interpretable-by-design text understanding with iteratively generated concept bottleneck.arXiv preprint arXiv:2310.19660. Niclas Luick. 2024. Universal response and emer- gence of induction in llms.arXiv preprint arXiv:2411.07071. Scott M Lundberg and Su-In Lee. 2017. A unifie...

    2024 arXiv

  16. [30]

    Alireza Makhzani and Brendan Frey

    Towards principled evaluations of sparse au- toencoders for interpretability and control.Preprint, arXiv:2405.08366. Alireza Makhzani and Brendan Frey. 2013. K-sparse autoencoders.arXiv preprint arXiv:1312.5663. Ana Marasovic, Iz Beltagy, Doug Downey, and Matthew Peters. 2022....

    2013 arXiv

  17. [32]

    Maximilian Mozes, Tolga Bolukbasi, Ann Yuan, Frederick Liu, Nithum Thain, and Lucas Dixon

    Saro: Enhancing llm safety through reasoning- based alignment.arXiv preprint arXiv:2504.09420. Maximilian Mozes, Tolga Bolukbasi, Ann Yuan, Frederick Liu, Nithum Thain, and Lucas Dixon

    arXiv

  18. [33]

    Anish Mudide, Joshua Engels, Eric J Michaud, Max Tegmark, and Christian Schroeder de Witt

    Gradient-based automated iterative re- covery for parameter-efficient tuning.Preprint, arXiv:2302.06598. Anish Mudide, Joshua Engels, Eric J Michaud, Max Tegmark, and Christian Schroeder de Witt. 2025. Ef- ficient dictionary learning with switch sparse autoen- coders. InThe Th...

    2025 arXiv

  19. [34]

    InFindings of the Association for Com- putational Linguistics: NAACL 2025, pages 1604– 1635, Albuquerque, New Mexico

    Decoding dark matter: Specialized sparse autoencoders for interpreting rare concepts in founda- tion models. InFindings of the Association for Com- putational Linguistics: NAACL 2025, pages 1604– 1635, Albuquerque, New Mexico. Association for Computational Linguistics. Neel Na...

    2025 arXiv

  20. [35]

    Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Tom Lieberum, Vikrant Varma, Janos Kramar, Rohin Shah, and Neel Nanda

    Refining input guardrails: Enhancing llm-as-a- judge efficiency through chain-of-thought fine-tuning and alignment.arXiv preprint arXiv:2501.13080. Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Tom Lieberum, Vikrant Varma, Janos Kramar, Rohin Shah, and Neel Nanda. 2024a...

    2019 arXiv

  21. [38]

    InProceedings of the 2022 Conference on Empirical Methods in Nat- ural Language Processing, pages 2078–2093, Abu Dhabi, United Arab Emirates

    Entailer: Answering questions with faithful and truthful chains of reasoning. InProceedings of the 2022 Conference on Empirical Methods in Nat- ural Language Processing, pages 2078–2093, Abu Dhabi, United Arab Emirates. Association for Com- putational Linguistics. Xue Tan, Hao...

    2022 arXiv

  22. [40]

    InICLR 2025 Workshop on Building Trust in Language Models and Applications

    Finding sparse autoencoder representations of errors in cot prompting. InICLR 2025 Workshop on Building Trust in Language Models and Applications. Che-Ping Tsai, Chih-Kuan Yeh, and Pradeep Ravikumar

    2025

  23. [41]

    InProceedings of the 37th Interna- tional Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY , USA

    Sample based explanations via generalized representers. InProceedings of the 37th Interna- tional Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY , USA. Curran Associates Inc. Yamei Tu, Jiayi Xu, and Han-Wei Shen. 2021. Key- wordmap: Attention-based...

    2021 arXiv

  24. [42]

    Dodrio: Exploring transformer models with interactive visualization. InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Con- ference on Natural Language Processing: System Demonstrations, pages 132–141, On...

    2024

  25. [43]

    InFindings of the Associa- tion for Computational Linguistics: EMNLP 2023, pages 2550–2575, Singapore

    Large language models are better reasoners with self-verification. InFindings of the Associa- tion for Computational Linguistics: EMNLP 2023, pages 2550–2575, Singapore. Association for Com- putational Linguistics. Sarah Wiegreffe and Yuval Pinter. 2019. Attention is not not e...

    2023 arXiv

  26. [44]

    InInternational Conference on Machine Learning, pages 24150–24176

    Davinz: Data valuation using deep neural net- works at initialization. InInternational Conference on Machine Learning, pages 24150–24176. PMLR. Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. 2024. LESS: Se- lecting influential data for targete...

    2024 arXiv

  27. [483]

    why should i trust you?

    IEEE. Zhongzheng Ren, Raymond Yeh, and Alexander Schwing. 2020. Not all unlabeled data are equal: Learning to weight data in semi-supervised learning. Advances in Neural Information Processing Systems, 33:21786–21797. Farnoush Rezaei Jafari, Grégoire Montavon, Klaus- Robert Mü...

    2020

  28. [2013]

    InProceedings of the 2013 conference of the north american chapter of the as- sociation for computational linguistics: Human lan- guage technologies, pages 746–751

    Linguistic regularities in continuous space word representations. InProceedings of the 2013 conference of the north american chapter of the as- sociation for computational linguistics: Human lan- guage technologies, pages 746–751. Aditi Mishra, Bretho Danzy, Utkarsh Soni, Anja...

    2013 arXiv

  29. [2019]

    Justin Theodorus, V Swaytha, Shivani Gautam, Adam Ward, Mahir Shah, Cole Blondin, and Kevin Zhu

    What do you learn from context? probing for sentence structure in contextualized word representa- tions.arXiv preprint arXiv:1905.06316. Justin Theodorus, V Swaytha, Shivani Gautam, Adam Ward, Mahir Shah, Cole Blondin, and Kevin Zhu

    1905 arXiv

  30. [2020]

    In Proceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 5553– 5563, Online

    Explaining black box predictions and unveil- ing data artifacts through influence functions. In Proceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 5553– 5563, Online. Association for Computational Lin- guistics. Michael Hanna, Olli...

  31. [2021]

    Logan Engstrom, Axel Feldmann, and Aleksander M ˛ adry

    A mathematical framework for transformer circuits.Transformer Circuits Thread, 1(1):12. Logan Engstrom, Axel Feldmann, and Aleksander M ˛ adry. 2024. Dsdm: model-aware dataset selection with datamodels. InProceedings of the 41st Interna- tional Conference on Machine Learning, ...

    2024 arXiv

  32. [2022]

    InInternational Conference on Learning Representations

    Resolving training biases via influence-based data relabeling. InInternational Conference on Learning Representations. János Kramár, Tom Lieberum, Rohin Shah, and Neel Nanda. 2024. Atp*: An efficient and scalable method for localizing llm behaviour to components.arXiv preprint...

    2024 arXiv

  33. [2023]

    InAdvances in Neural Infor- mation Processing Systems, volume 36, pages 16318– 16352

    Towards automated circuit discovery for mech- anistic interpretability. InAdvances in Neural Infor- mation Processing Systems, volume 36, pages 16318– 16352. Curran Associates, Inc. R Dennis Cook and Sanford Weisberg. 1980. Char- acterizations of an empirical influence functio...

    1980

  34. [2024]

    Samira Abnar and Willem Zuidema

    Are you still on track!? catching llm task drift with activations.arXiv preprint arXiv:2406.00799. Samira Abnar and Willem Zuidema. 2020. Quantify- ing attention flow in transformers. InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pa...

    2020 arXiv

  35. [2025]

    Sahar Abdelnabi, Aideen Fay, Giovanni Cherubin, Ahmed Salem, Mario Fritz, and Andrew Paverd

    SAFE: A sparse autoencoder-based frame- work for robust query enrichment and hallucination mitigation in llms.arXiv preprint arXiv:2503.03032. Sahar Abdelnabi, Aideen Fay, Giovanni Cherubin, Ahmed Salem, Mario Fritz, and Andrew Paverd

    arXiv

  36. [3328]

    Aaquib Syed, Can Rager, and Arthur Conmy

    PMLR. Aaquib Syed, Can Rager, and Arthur Conmy. 2024. Attribution patching outperforms automated circuit discovery. InProceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Net- works for NLP, pages 407–416, Miami, Florida, US. Association for Computati...

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.