Pith. sign in

REVIEW 5 major objections 5 minor 2 cited by

MIB: A Mechanistic Interpretability Benchmark

T0 review · 5 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper introduces MIB, a two-track benchmark for mechanistic interpretability, and reports that attribution and mask optimization methods lead circuit localization while supervised DAS leads causal variable localization.

desk verdict A credible, field-shaping benchmark whose circuit track is solid; the causal-variable findings are real but rest on high-level causal models the paper never independently validates. read the letter →

arxiv 2504.13151 v2 pith:OOWKI5UA submitted 2025-04-17 cs.LG cs.AIcs.CL

Aaron Mueller , Atticus Geiger , Sarah Wiegreffe , Dana Arad , Iván Arcuschin , Adam Belfki , Yik Siu Chan , Jaden Fiotto-Kaufman
show 15 more authors
Tal Haklay Michael Hanna Jing Huang Rohan Gupta Yaniv Nikankin Hadas Orgad Nikhil Prakash Anja Reusch Aruna Sankaranarayanan Shun Shao Alessandro Stolfo Martin Tutek Amir Zur David Bau Yonatan Belinkov
This is my paper · ORCID
classification cs.LGcs.AIcs.CL
keywords mechanisticinterpretabilitybenchmarkcircuitlocalizationcausalvariableactivationpatchingsparseautoencodersdistributedalignmentsearchinterchangeinterventions
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MIB is a benchmark that tries to settle a recurring question in mechanistic interpretability: when a new method claims to find the circuit or the concept a language model uses, how do we know it is actually better? It does this by defining two tasks—locating the model components and connections that drive a behavior, and locating the hidden-vector features that implement a specific causal variable—across four tasks and five models, with fixed counterfactual inputs, standardized metrics, and private test sets. If the benchmark is right, the field gains a shared yardstick: a method can be judged by how precisely and concisely it recovers causal pathways or causal variables, rather than by ad hoc comparisons. The paper's headline findings are that attribution and mask-optimization methods perform best on circuit localization, and that supervised distributed alignment search (DAS) outperforms unsupervised featurizers, with sparse autoencoder features no better than raw neuron dimensions for localizing known causal variables.

What carries the argument

The load-bearing mechanism is the counterfactual interchange intervention, applied in two ways. For circuits, activation patching—replacing a component's activation with the value it would take on a counterfactual input—produces a faithfulness curve across circuit sizes, summarized by two new metrics: circuit performance ratio (CPR, area under the faithfulness curve) and circuit-model distance (CMD, area between the faithfulness curve and perfect faithfulness). For causal variables, the same intervention is applied to aligned features of a hidden vector, and the metric is interchange intervention accuracy (IIA): the fraction of cases where patching the features has the same output effect as patching the variable in a high-level causal model. The featurizers compared—identity (full vector), PCA, sparse autoencoders, and DAS—are all evaluated through this shared intervention lens.

What would settle it

Train a transformer with a hand-verified ground-truth circuit on a second task and check whether the method ranking by CPR and CMD matches the ranking by edge-level precision and recall; divergence would show the benchmark metrics do not measure true circuit recovery.

Watch

Extended reading notes

Core claim

The central claim is that MIB is a valid, useful benchmark for comparing mechanistic interpretability methods across method families, and that using it reveals a clear differentiation between methods. On the circuit localization track, edge attribution patching with integrated gradients over inputs (EAP-IG-inputs) with counterfactual ablations generally achieves the lowest circuit-model distance and highest circuit performance ratio; on the causal variable localization track, DAS, the supervised featurizer, consistently gives the most faithful alignments, while masks learned on PCA or SAE features do not beat masks learned on ordinary hidden-unit dimensions. The paper also claims these findings increase confidence that real progress has been made in the field.

Load-bearing premise

The causal variable track presumes that the high-level mental model it tests (like 'carry the one' in addition or 'find the answer position' in multiple choice) is actually the algorithm the network implements; if that variable is not really computed by the model, the alignment scores cannot tell us about real mechanisms.

Editorial extensions

If this is right

  • Circuit discovery methods can now be ranked by CPR and CMD instead of single-circuit faithfulness, and methods that optimize for sparsity plus faithfulness, like EAP-IG, set a strong default baseline.
  • SAE features should not be assumed to be better units than neurons for causal localization; any new featurizer must beat DBM-on-neurons to claim progress.
  • DAS provides a skyline for supervised alignment, and unsupervised methods are measured against it.
  • The benchmark's private test sets and public leaderboards allow future methods to be compared on unseen data, reducing hill-climbing on known datasets.
  • The carry-the-one causal variable in two-digit addition remains an open target, since current baselines are near chance; beating them would constitute genuine progress.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the causal-variable results generalize, they put pressure on the common assumption that sparse autoencoders are the right unit of analysis for mechanistic interpretability; a similar pattern already appears in steering benchmarks, where simple directions beat SAE features.
  • A natural extension beyond the paper is to test whether EAP-IG's advantage persists when circuits are built greedily rather than by top-k edge selection, since the paper uses top-k for most methods.
  • The near-chance carry-the-one results suggest the benchmark could be extended to non-linear feature spaces, such as onion representations, as a next testbed; the paper notes this possibility implicitly.
  • One could use the same two-track design on non-language modalities, since the paper limits itself to language models but the metrics are modality-agnostic.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes MIB, a two-track benchmark for mechanistic interpretability. The circuit localization track introduces integrated circuit performance ratio (CPR) and circuit-model distance (CMD) metrics that marginalize over circuit size, and compares random, activation patching, attribution patching (EAP/NAP with and without integrated gradients), information flow routes, and uniform gradient sampling on IOI, arithmetic, MCQA, and ARC across Llama-3.1 8B, Gemma-2 2B, Qwen-2.5 0.5B, and GPT-2 Small, plus an InterpBench ground-truth circuit model. The causal variable localization track uses interchange intervention accuracy (IIA) to evaluate alignments of high-level causal variables (XCarry, XOrder, OAnswer, STok, SPos, and RAVEL attributes) to hidden-vector features, comparing DAS, DBM on raw dimensions, DBM on PCA and SAE features, and full-vector baselines. The paper's central claims are that attribution and mask optimization methods perform best on circuit localization and that supervised DAS performs best on causal variable localization, while SAE features are not better than raw neurons. The paper also describes public and private leaderboards for future submissions.

Significance. Conditional on the validity of the high-level causal models, MIB would be a valuable community resource: it spans four tasks and five model classes, provides standardized counterfactual datasets with public/private splits, introduces CPR/CMD metrics that avoid a circuit-size threshold, includes an InterpBench AUROC control for ground-truth circuits, and proposes two public leaderboards for cross-method comparison. The comparative baseline sweep is substantial, and the headline findings are falsifiable claims of the kind the field needs. However, the causal variable track's construct validity is the load-bearing issue: if the hypothesized variables are not implemented by the evaluated models, IIA-based rankings measure fit to a speculative abstraction rather than localization of real mechanisms. The lack of uncertainty quantification in the circuit localization tables and the absence of private-test results for the causal variable track further weaken the comparative claims as currently stated.

major comments (5)
  1. [§4.1 and App. F.5.2] The causal variable track's interpretation presupposes that the high-level causal models H are faithful abstractions of the evaluated networks, and App. B itself states that 'these graphs may not always exist in the models we evaluate in the exact forms shown here.' The arithmetic results show that this is not a merely hypothetical concern: App. F.5.2 reports that the initial apparent XCarry alignment in Llama was 'likely a representation of the tens-digit output, which perfectly correlates with the ones-carry variable in these counterfactuals,' and that with random counterfactuals the method fails completely. Because IIA is computed only on counterfactual types correlated with the target variable, the DAS-versus-SAE ranking in Table 3(b), and by analogy the other causal variable tables, may measure fit to a hypothesized variable rather than localization of a mechanism the model actually implements. I ask the authors to add negative-control counterfactual sets for every task-variable pair (e.g., random counterfactuals that decorrelate the target variable from output-relevant features) and to report IIA on those controls, or to explicitly restrict the causal variable ranking claims to task-variable pairs that pass such controls.
  2. [§4.6 and Figure 14] The IOI causal model HIOI is fitted to the same model being evaluated: the paper states that after replicating Wang et al.'s experiments, the authors 'fit our own linear model' to GPT-2's logit differences, yielding the coefficients 0.048 + 2.005*PositionSignal + 0.768*TokenSignal. Evaluation of STok and SPos alignments against this fitted target is therefore partly circular: the 'high-level' target is a compressed description of the exact model and dataset used in Table 3(d), not an independent causal hypothesis. At minimum, the authors should specify whether the linear coefficients were fixed before benchmarking, and ideally evaluate on a held-out model or a pre-registered causal model. Without this, the MSE values in Table 3(d) primarily measure regression fit rather than the faithfulness of an independently specified abstraction.
  3. [§3.3, Tables 2 and 14–16] The headline circuit-localization claims ('EAP-IG-inputs achieves the highest performance on average') are made without error bars, significance tests, or sensitivity analysis, despite differences as small as 0.00–0.07 between methods in several columns (e.g., Table 2, ARC (E) Gemma: EAP-IG-inputs CMD 0.14 versus EAP-IG-activations 0.13; IOI Qwen CMD 0.02 versus 0.01). Many entries appear to be single runs; only the random baseline is averaged over three seeds. This does not establish that the best method is distinguishable from close competitors. I ask the authors to add bootstrap confidence intervals over examples or seeds, report variance for all methods, and, where possible, run pairwise significance tests or at least report effect sizes with intervals.
  4. [§4.1 and App. F.5.3] The benchmark filters out all examples where the model is incorrect on the base input or on any counterfactual input before computing IIA. This is a post-hoc selection that can inflate IIA and change method rankings, particularly in the low-data MCQA and ARC settings. App. F.5.3 explicitly states 'We filter out failure cases' for RAVEL after reporting 66.5–70.5% model accuracy. The paper should report results on the unfiltered set alongside the filtered set, or provide evidence that the filtering does not drive the DAS advantage over unsupervised featurizers. As written, the scores may reflect selection of easy examples rather than the quality of the localization methods.
  5. [§2.4 and §4] No private-test results are reported for the causal variable localization track. The circuit localization track includes private-test CMD and CPR tables (App. D, Tables 15–16), but the causal variable track reports only Table 3, which is computed on the same public counterfactual distributions used to train DBM and DAS. The causal variable rankings may therefore reflect overfitting to the particular counterfactual types, and the paper's claim that MIB enables leaderboard evaluation on a private test set is not demonstrated for this track. Either add private-test causal variable results, or explicitly define held-out counterfactual types as the evaluation protocol and report those results.
minor comments (5)
  1. [§3.1] In the paragraph introducing the metrics, there is a typo: 'we propose to to quantify CPR' should read 'we propose to quantify CPR'.
  2. [Table 2 caption] The caption contains a typo: 'Webold and underline' should read 'We bold and underline'.
  3. [§2 and App. C.6] The paper refers to 'this HuggingFace URL', 'this GitHub URL', and 'this HuggingFace URL' as placeholders; the actual URLs should be included in the final version so that the promised datasets, code, and leaderboards are accessible.
  4. [App. F.5.3] The sentence 'The dataset contains many cities, so considering that there are many countries and languages to choose from, this is good performance' is awkwardly phrased and should be rewritten for clarity.
  5. [Table 4] The notation table defines 'c' as 'An counterfactual input'; the article 'An' should be 'A'.

Circularity Check

1 steps flagged · score 4.0 of 10

Partial circularity in the IOI track's fitted causal model; the overall benchmark comparison retains independent content.

  1. fitted input called prediction [§4.6 (Indirect Object Identification), App. F.5.4]
    "we replicate these experiments on our curated datasets and fit our own linear model that we use to define a causal model HIOI that predicts the logit difference between the indirect object and the subject. ... finding that 0.048 + 2.005PositionSignal + 0.768TokenSignal is the best predictor of logit difference."

    The target of the IOI causal-variable evaluation is not an independent ground truth: HIOI's output mechanism is a least-squares regression fit to the LM's own logit differences on the same counterfactual types (IO↔S1 Flip, IO↔S2 Flip, and their composition) that are later used for evaluation. Table 3d scores alignments by 'mean-squared error between the causal model logit and the deep learning model logit,' i.e., agreement with this fitted regression. DAS is trained to minimize exactly this MSE (App. F.3), so its top IOI score is partly forced by construction. The paper's own arithmetic check (App.

full rationale

The circuit localization track is self-contained: CPR and CMD are explicit aggregations of faithfulness, methods are evaluated on held-out counterfactual test sets, and the InterpBench AUROC provides an external ground-truth check. This part shows no circularity. The causal variable localization track for MCQA, ARC, and RAVEL uses high-level causal models as explicit hypotheses, and the fact that DAS and DBM are trained to maximize the evaluation metric is acknowledged ('DAS is the only method that learns features with supervision from the high-level causal model, so it is not surprising this method performs best'). Supervised optimization against a benchmark objective is a standard benchmark design, not a hidden circularity. The one genuine partial-circularity step is IOI: the high-level model's output is a linear regression fit to the same LM's logit differences on the same counterfactual types used for scoring, so the evaluation target is partly a fitted description of the LM, and DAS's advantage on that task is partly by construction. This does not invalidate the whole benchmark, because the SAE-versus-neuron comparison and the attribution-method circuit results have independent content and are anchored by external benchmarks such as InterpBench, RAVEL, and SAEBench. The App. B admission that 'these graphs may not always exist in the models we evaluate in the exact forms shown here' is a validity limitation rather than a circularity, and the arithmetic track's honest reporting of the confounded XCarry result further supports that the benchmark can detect failures. Overall, one prediction reduces partly to a fit, while the central claims retain independent evidentiary weight.

Assumptions & free parameters 7 free parameters · 4 assumptions · 3 invented entities

The benchmark's causal variable track rests on assumed high-level causal models, several of which are not independently verified. DAS and IOI introduce fitted parameters that influence results. The free parameters listed are the main tuning choices the comparisons depend on.

free parameters (7)
  • DAS dimensionality per task = 16 (XOrder, XCarry), 32 (STok, SPos), half residual stream (OAnswer), eighth residual stream (RAVEL attributes)
    Chosen by hand for each causal variable and not swept; the benchmark's comparative claims depend on these settings.
  • DBM temperature schedule = 1.0 to 0.01
    Chosen ad hoc for mask training; may affect SAE vs neuron comparison.
  • UGS regularization lambda = 1e-3
    Selected via validation hyperparameter search over 1e-2 to 1e-7; affects circuit localization results for UGS.
  • AP-IG integration steps Z = 5
    Taken from Hanna et al. 2024, not tuned; affects approximation quality of attribution methods.
  • Learning rates for DAS/DBM = 0.01 standard, 1.0 for IOI
    Manually chosen hyperparameters for training featurizers; results may shift with tuning.
  • Optimal ablation training settings = lr 1e-3, batch 20, up to 1000 steps
    Manual choices for learning ablation vectors; affect EAP-OA baseline.
  • HIOI linear model coefficients = intercept 0.048, position 2.005, token 0.768
    Fit to GPT-2's logit differences on the authors' IOI dataset; the causal model is therefore derived from the LM being explained, making IOI evaluation partially circular.
assumptions (4)
  • domain assumption High-level causal models HMCQA, H+, HRAVEL, HIOI are faithful abstractions of LM computations
    The causal variable track evaluates alignments to these models; if the variables are not mechanistically implemented, alignment scores lose meaning (App. B admits this).
  • domain assumption The chosen faithfulness metrics (logit-difference ratio and interchange intervention accuracy) capture what it means for a circuit or variable to explain model behavior
    Metrics define the task; authors follow Zhang & Nanda (2024) but provide no theoretical guarantee that these metrics correspond to ground-truth mechanisms (though InterpBench AUROC provides a partial check).
  • domain assumption Counterfactual inputs for each task correctly isolate the task-relevant causal variables
    All evaluations use interchange interventions with these counterfactuals; biased counterfactuals would distort all results (Section 2.2).
  • standard math Standard transformer and gradient mathematics
    Background for attribution patching, DAS, and interventions.
invented entities (3)
  • XCarry (carry-the-one variable) in H+
    purpose: High-level variable for two-digit arithmetic evaluation
    Hypothesized to exist in Llama/Gemma; benchmark results are at chance for Gemma, suggesting it may not be linearly represented. No external falsifiable prediction.
  • XOrder and OAnswer in HMCQA
    purpose: Variables for answer position and answer token in multiple-choice tasks
    Hypothesized based on prior work; DAS results provide internal evidence but no external prediction.
  • STok and SPos in HIOI
    purpose: Subject token and position variables for IOI
    Based on Wang et al. 2023; the causal model is fitted to the authors' dataset with linear regression, so it is partly derived from the data it evaluates.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MIB: A Mechanistic Interpretability Benchmark." pith.science (2026). https://pith.science/paper/OOWKI5UA

@misc{pith2026250413151,
  author       = {Pith},
  title        = {Pith review of: MIB: A Mechanistic Interpretability Benchmark},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OOWKI5UA}},
  note         = {Machine review of arXiv:2504.13151}
}
read the original abstract

How can we know whether new mechanistic interpretability methods achieve real improvements? In pursuit of lasting evaluation standards, we propose MIB, a Mechanistic Interpretability Benchmark, with two tracks spanning four tasks and five models. MIB favors methods that precisely and concisely recover relevant causal pathways or causal variables in neural language models. The circuit localization track compares methods that locate the model components - and connections between them - most important for performing a task (e.g., attribution patching or information flow routes). The causal variable localization track compares methods that featurize a hidden vector, e.g., sparse autoencoders (SAEs) or distributed alignment search (DAS), and align those features to a task-relevant causal variable. Using MIB, we find that attribution and mask optimization methods perform best on circuit localization. For causal variable localization, we find that the supervised DAS method performs best, while SAE features are not better than neurons, i.e., non-featurized hidden vectors. These findings illustrate that MIB enables meaningful comparisons, and increases our confidence that there has been real progress in the field.

Figures

Figures reproduced from arXiv: 2504.13151 by the authors.

Figure 1
Figure 1. An overview of MIB. We compare different circuit (§3) and causal variable (§4) localization methods on their ability to faithfully represent a model’s behavior on a given task. We provide standardized datasets and metrics for this purpose, and accept user submissions for display on two public leaderboards (§2). of their behavior. To this end, mechanistic interpretability (MI) methods have proliferated quickly. MI me… view at source ↗
Figure 2
Figure 2. Definition of our faithfulness metrics. CPR, in orange, is the area under the faithfulness curve (the black line); it captures how well the method finds performant circuits at many circuit sizes. CMD, in blue, is the area between the faithfulness curve and the line at f = 1; it captures how closely the circuit’s behavior resembles the model’s task-specific behavior at many circuit sizes. Because we define f as a rat… view at source ↗
Figure 3
Figure 3. A schematic of the causal variable localization track submission and evaluation. Users submit an alignment between a high-level causal variable X and hidden vector features ΠX in an LM (top). In evaluations, aligned interchange interventions are performed with base and counterfactual inputs (b, c) on the high-level causal model HX←Get(H(c),X)(b) and the low-level neural network NΠX←Get(N(c),ΠX)(b). The more similar … view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: An IOI example. Each input is paired with a set of templatically generated counterfactuals. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_4.png]
Figure 5
Figure 5. Figure 5: Leaderboard for the circuit localization track [PITH_FULL_IMAGE:figures/full_fig_p026_5.png]
Figure 6
Figure 6. Figure 6: Leaderboard for the causal variable localization track. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_6.png]
Figure 7
Figure 7. Figure 7: Causal model for multiple choice question answering. The model operates through a two-step mechanism: first, the TextPrompt is processed to generate an AnswerPointer that identifies the position of the correct answer in the options list. Second, this AnswerPointer is u…
Figure 9
Figure 9. Figure 9: Causal model for two-digit addition arithmetic. The model processes addition through a series of interdependent mechanisms: (1) The 1’s digits from both addends directly influence the output 1’s digit through modular addition. (2) When the sum of 1’s digits exceeds 9, …
Figure 11
Figure 11. Figure 11: Causal model for the RAVEL task. Given a prompt querying an attribute of a city entity, the model extracts the entity and queried attribute as input variables, then identifies the values of the ACont, ACountry , and ALang attributes for the entity. Lastly, it decides …
Figure 12
Figure 12. Figure 12: Interchange Intervention Accuracy (IIA) at each layer of Gemma-2 2B, when targeting the ACountry variable using the attribute counterfactual. The Full Vector baseline fails to isolate the target variable, whereas DAS achieves high accuracy in the mid layers. (a) IIA u…
Figure 13
Figure 13. Figure 13: Interchange Intervention Accuracy (IIA) at each layer of Gemma-2 2B when targeting the ACountry variable using the Wikipedia counterfactual. This counterfactual presents a more challenging setting. The Full Vector baseline fails to disentangle the attributes, while DA…
Figure 14
Figure 14. Figure 14: Causal model for indirect object identification. The TextPrompt is processed to extract the subject token STok and the subject position SPos. The output variable mechanism (1) compares the token and position to the input, and determines whether the token and position …

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Temporal Preference Concepts and their Functions in a Large Language Model

    cs.LG 2026-05 unverdicted novelty 6.5 of 10

    Temporal preference in Qwen3-4B-Instruct-2507 localizes to layers 17–35 (especially L24 attention), has curved residual-stream geometry, is behaviorally unstable, and can be bidirectionally steered.

  2. Correcting Gradient-Based Circuit Localization via Interaction-Aware Backpropagation

    cs.CL 2025-05 conditional novelty 6.0 of 10

    GIM modifies softmax gradients with a temperature adjustment, layer-norm freeze, and gradient normalization to counter attention self-repair, improving the faithfulness of gradient-based LLM attributions.

Reference graph

Works this paper leans on

80 extracted references · 30 canonical work pages · cited by 2 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    D., D'Oosterlinck, K., Feder, A., Gat, Y

    Abraham, E. D., D'Oosterlinck, K., Feder, A., Gat, Y. O., Geiger, A., Potts, C., Reichart, R., and Wu, Z. CEB ab: Estimating the causal effects of real-world concepts on NLP model behavior. In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K. (eds.), Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/forum?id=3AbigH4s-ml

    work page 2022

  3. [3]

    Naturalistic causal probing for morpho-syntax

    Amini, A., Pimentel, T., Meister, C., and Cotterell, R. Naturalistic causal probing for morpho-syntax. Transactions of the Association for Computational Linguistics, 11: 0 384--403, 2023. doi:10.1162/tacl_a_00554. URL https://aclanthology.org/2023.tacl-1.23/

    doi:10.1162/tacl_a_00554 2023

  4. [4]

    Causal G ym: Benchmarking causal interpretability methods on linguistic tasks

    Arora, A., Jurafsky, D., and Potts, C. Causal G ym: Benchmarking causal interpretability methods on linguistic tasks. In Ku, L., Martins, A., and Srikumar, V. (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thailand, August 11-16, 2024 , pp.\ 14638--14663. Associat...

    doi:10.18653/v1/2024.acl-long.785 2024

  5. [5]

    G., and Augenstein, I

    Atanasova, P., Camburu, O.-M., Lioma, C., Lukasiewicz, T., Simonsen, J. G., and Augenstein, I. Faithfulness tests for natural language explanations. In Rogers, A., Boyd-Graber, J., and Okazaki, N. (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp.\ 283--294, Toronto, Canada, July 2...

    doi:10.18653/v1/2023.acl-short.25 2023

  6. [6]

    E., Hume, T., Carter, S., Henighan, T., and Olah, C

    Bricken, T., Templeton, A., Batson, J., Chen, B., Jermyn, A., Conerly, T., Turner, N., Anil, C., Denison, C., Askell, A., Lasenby, R., Wu, Y., Kravec, S., Schiefer, N., Maxwell, T., Joseph, N., Hatfield-Dodds, Z., Tamkin, A., Nguyen, K., McLean, B., Burke, J. E., Hume, T., Carter, S., Henighan, T., and Olah, C. Towards monosemanticity: Decomposing languag...

    work page 2023

  7. [7]

    Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A.,...

    1901

  8. [8]

    D., Schlichtkrull, M

    Cao, N. D., Schlichtkrull, M. S., Aziz, W., and Titov, I. How do decisions emerge across layers in neural models? I nterpretation with differentiable masking. In Webber, B., Cohn, T., He, Y., and Liu, Y. (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp.\ 3243--3255, Online, November 2020. Associati...

    doi:10.18653/v1/2020.emnlp-main.262 2020

Show all 80 references
  1. [9]

    D., Schmid, L., Hupkes, D., and Titov, I

    Cao, N. D., Schmid, L., Hupkes, D., and Titov, I. Sparse interventions in language models with differentiable masking. In Proceedings of the Fifth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, 2022. URL https://doi.org/10.18653/v1/2022.blackboxnlp-1.2

    2022 doi

  2. [10]

    Causal scrubbing, a method for rigorously testing interpretability hypotheses

    Chan, L., Garriga-Alonso, A., Goldwosky-Dill, N., Greenblatt, R., Nitishinskaya, J., Radhakrishnan, A., Shlegeris, B., and Thomas, N. Causal scrubbing, a method for rigorously testing interpretability hypotheses. AI Alignment Forum, 2022. https://www.alignmentforum.org/posts/J...

    2022

  3. [11]

    and Geiger, A

    Chaudhary, M. and Geiger, A. Evaluating open-source sparse autoencoders on disentangling factual knowledge in GPT -2 small. CoRR, abs/2409.04478, 2024. URL https://arxiv.org/abs/2409.04478

    2024 arXiv

  4. [12]

    Think you have solved question answering? T ry ARC , the AI2 R easoning C hallenge

    Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., and Tafjord, O. Think you have solved question answering? T ry ARC , the AI2 R easoning C hallenge. arXiv preprint arXiv:1803.05457, 2018. URL https://arxiv.org/abs/1803.05457

    2018 arXiv

  5. [13]

    Evaluating the ripple effects of knowledge editing in language models

    Cohen, R., Biran, E., Yoran, O., Globerson, A., and Geva, M. Evaluating the ripple effects of knowledge editing in language models. Transactions of the Association for Computational Linguistics, 12: 0 283--298, 2024. doi:10.1162/tacl_a_00644. URL https://aclanthology.org/2024....

    2024 doi

  6. [14]

    Towards automated circuit discovery for mechanistic interpretability

    Conmy, A., Mavor-Parker, A., Lynch, A., Heimersheim, S., and Garriga-Alonso, A. Towards automated circuit discovery for mechanistic interpretability. Advances in Neural Information Processing Systems, 36: 0 16318--16352, 2023

    2023

  7. [15]

    Are neural nets modular? I nspecting functional modularity through differentiable weight masks

    Csord \'a s, R., van Steenkiste, S., and Schmidhuber, J. Are neural nets modular? I nspecting functional modularity through differentiable weight masks. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=7uVcpu-gMD

    2021

  8. [16]

    D., and Geiger, A

    Csord \'a s, R., Potts, C., Manning, C. D., and Geiger, A. Recurrent neural networks learn to store and generate sequences using non-linear representations. In The 7th BlackboxNLP Workshop, 2024. URL https://openreview.net/forum?id=NUQeYgg8x4

    2024

  9. [17]

    Representational analysis of binding in language models

    Dai, Q., Heinzerling, B., and Inui, K. Representational analysis of binding in language models. In Al-Onaizan, Y., Bansal, M., and Chen, Y.-N. (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp.\ 17468--17493, Miami, Florida, USA...

    2024 doi

  10. [18]

    R., and Bau, D

    Davies, X., Nadeau, M., Prakash, N., Shaham, T. R., and Bau, D. Discovering variable binding circuitry with desiderata. CoRR, abs/2307.03637, 2023. doi:10.48550/ARXIV.2307.03637. URL https://doi.org/10.48550/arXiv.2307.03637

    2023 doi arXiv

  11. [19]

    The L lama 3 herd of models

    Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The L lama 3 herd of models. CoRR, abs/2407.21783, 2024. URL https://arxiv.org/abs/2407.21783

    2024 arXiv

  12. [20]

    and Voita, E

    Ferrando, J. and Voita, E. Information flow routes: Automatically interpreting language models at scale. In Al-Onaizan, Y., Bansal, M., and Chen, Y.-N. (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp.\ 17432--17445, Miami, Flo...

    2024 doi

  13. [21]

    Ferrando, J., Sarti, G., Bisazza, A., and Costa - juss \` a , M. R. A primer on the inner workings of transformer-based language models. CoRR, abs/2405.00208, 2024. doi:10.48550/ARXIV.2405.00208. URL https://doi.org/10.48550/arXiv.2405.00208

    2024 doi arXiv

  14. [22]

    Causal analysis of syntactic agreement mechanisms in neural language models

    Finlayson, M., Mueller, A., Gehrmann, S., Shieber, S., Linzen, T., and Belinkov, Y. Causal analysis of syntactic agreement mechanisms in neural language models. In Zong, C., Xia, F., Li, W., and Navigli, R. (eds.), Proceedings of the 59th Annual Meeting of the Association for ...

    2021 doi

  15. [23]

    Neural natural language inference models partially embed theories of lexical entailment and negation

    Geiger, A., Richardson, K., and Potts, C. Neural natural language inference models partially embed theories of lexical entailment and negation. In Alishahi, A., Belinkov, Y., Chrupa a, G., Hupkes, D., Pinter, Y., and Sajjad, H. (eds.), Proceedings of the Third BlackboxNLP Work...

    2020 doi

  16. [24]

    Causal abstractions of neural networks

    Geiger, A., Lu, H., Icard, T., and Potts, C. Causal abstractions of neural networks. In Ranzato, M., Beygelzimer, A., Dauphin, Y. N., Liang, P., and Vaughan, J. W. (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing ...

    2021

  17. [25]

    Causal abstraction: A theoretical foundation for mechanistic interpretability

    Geiger, A., Ibeling, D., Zur, A., Chaudhary, M., Chauhan, S., Huang, J., Arora, A., Wu, Z., Goodman, N., Potts, C., and Icard, T. Causal abstraction: A theoretical foundation for mechanistic interpretability. CoRR, abs/2301.04709, 2024 a . URL https://arxiv.org/abs/2301.04709

    2024 arXiv

  18. [26]

    Geiger, A., Wu, Z., Potts, C., Icard, T., and Goodman, N. D. Finding alignments between interpretable causal variables and distributed neural representations. In Locatello, F. and Didelez, V. (eds.), Causal Learning and Reasoning, 1-3 April 2024, Los Angeles, California, USA ,...

    2024

  19. [27]

    Interp B ench: Semi-synthetic transformers for evaluating mechanistic interpretability techniques

    Gupta, R., Arcuschin, I., Kwa, T., and Garriga-Alonso, A. Interp B ench: Semi-synthetic transformers for evaluating mechanistic interpretability techniques. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024. URL https:/...

    2024

  20. [28]

    Have faith in faithfulness: Going beyond circuit overlap when finding model mechanisms

    Hanna, M., Pezzelle, S., and Belinkov, Y. Have faith in faithfulness: Going beyond circuit overlap when finding model mechanisms. In ICML 2024 Workshop on Mechanistic Interpretability, 2024. URL https://openreview.net/forum?id=grXgesr5dT

    2024

  21. [29]

    Llama S cope: Extracting millions of features from llama-3.1-8b with sparse autoencoders

    He, Z., Shu, W., Ge, X., Chen, L., Wang, J., Zhou, Y., Liu, F., Guo, Q., Huang, X., Wu, Z., Jiang, Y., and Qiu, X. Llama S cope: Extracting millions of features from llama-3.1-8b with sparse autoencoders. CoRR, abs/2410.20526, 2024. doi:10.48550/ARXIV.2410.20526. URL https://d...

    doi arXiv

  22. [30]

    We can't understand AI using our existing vocabulary

    Hewitt, J., Geirhos, R., and Kim, B. We can't understand AI using our existing vocabulary. CoRR, abs/2502.07586, 2025. URL https://arxiv.org/abs/2502.07586

    2025 arXiv

  23. [31]

    RAVEL : Evaluating interpretability methods on disentangling language model representations

    Huang, J., Wu, Z., Potts, C., Geva, M., and Geiger, A. RAVEL : Evaluating interpretability methods on disentangling language model representations. In Ku, L.-W., Martins, A., and Srikumar, V. (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational L...

    2024 doi

  24. [32]

    Unified view of grokking, double descent and emergent abilities: A comprehensive study on algorithm task

    Huang, Y., Hu, S., Han, X., Liu, Z., and Sun, M. Unified view of grokking, double descent and emergent abilities: A comprehensive study on algorithm task. In First Conference on Language Modeling, 2024 b . URL https://openreview.net/forum?id=cG1EbmWiSs

    2024

  25. [33]

    R., Ewart, A., and Sharkey, L

    Huben, R., Cunningham, H., Smith, L. R., Ewart, A., and Sharkey, L. Sparse autoencoders find highly interpretable features in language models. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=F76bwRSLeK

    2024

  26. [34]

    Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D

    Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., de las Casas, D., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., Lavaud, L. R., Lachaux, M.-A., Stock, P., Scao, T. L., Lavril, T., Wang, T., Lacroix, T., and Sayed, W. E. Mistral 7b. CoRR, abs/2310...

    arXiv

  27. [35]

    MRKL systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning

    Karpas, E., Abend, O., Belinkov, Y., Lenz, B., Lieber, O., Ratner, N., Shoham, Y., Bata, H., Levine, Y., Leyton-Brown, K., et al. MRKL systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning. arXiv ...

    2022 arXiv

  28. [36]

    S A E B ench: A comprehensive benchmark for sparse autoencoders, 2025

    Karvonen, A., Rager, C., Lin, J., Tigges, C., Bloom, J., Chanin, D., Lau, Y.-T., Farrell, E., Conmy, A., McDougall, C., Ayonrinde, K., Wearden, M., Marks, S., and Nanda, N. S A E B ench: A comprehensive benchmark for sparse autoencoders, 2025. URL https://www.neuronpedia.org/sae-bench

    2025

  29. [37]

    and Janson, L

    Li, M. and Janson, L. Optimal ablation for interpretability. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview.net/forum?id=opt72TYzwZ

    2024

  30. [38]

    and Gao, Y

    Li, R. and Gao, Y. Anchored answers: Unravelling positional bias in GPT -2's multiple-choice questions. arXiv preprint arXiv:2405.03205, 2024. URL https://arxiv.org/abs/2405.03205

    2024 arXiv

  31. [40]

    Does circuit analysis interpretability scale? evidence from multiple choice capabilities in chinchilla

    Lieberum, T., Rahtz, M., Kramár, J., Nanda, N., Irving, G., Shah, R., and Mikulik, V. Does circuit analysis interpretability scale? evidence from multiple choice capabilities in chinchilla. CoRR, abs/2307.09458, 2023 b . URL https://arxiv.org/abs/2307.09458

    2023 arXiv

  32. [41]

    Gemma S cope: Open sparse autoencoders everywhere all at once on G emma 2

    Lieberum, T., Rajamanoharan, S., Conmy, A., Smith, L., Sonnerat, N., Varma, V., Kramar, J., Dragan, A., Shah, R., and Nanda, N. Gemma S cope: Open sparse autoencoders everywhere all at once on G emma 2. In Belinkov, Y., Kim, N., Jumelet, J., Mohebbi, H., Mueller, A., and Chen,...

    2024 doi

  33. [42]

    J., and Tegmark, M

    Liu, Z., Michaud, E. J., and Tegmark, M. Omnigrok: Grokking beyond algorithmic data. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=zDiHoIWa0q1

    2023

  34. [43]

    and Tegmark, M

    Marks, S. and Tegmark, M. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets. In First Conference on Language Modeling, 2024. URL https://openreview.net/forum?id=aajyHYjjsk

    2024

  35. [44]

    J., Belinkov, Y., Bau, D., and Mueller, A

    Marks, S., Rager, C., Michaud, E. J., Belinkov, Y., Bau, D., and Mueller, A. Sparse feature circuits: Discovering and editing interpretable causal graphs in language models. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.ne...

    2025

  36. [45]

    J., and Belinkov, Y

    Meng, K., Bau, D., Andonian, A. J., and Belinkov, Y. Locating and editing factual associations in GPT . In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K. (eds.), Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/forum?id=-h6WAS6eE4

    2022

  37. [46]

    Circuit component reuse across tasks in transformer language models

    Merullo, J., Eickhoff, C., and Pavlick, E. Circuit component reuse across tasks in transformer language models. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net, 2024. URL https://openreview.net/fo...

    2024

  38. [47]

    Transformer circuit evaluation metrics are not robust

    Miller, J., Chughtai, B., and Saunders, W. Transformer circuit evaluation metrics are not robust. In First Conference on Language Modeling, 2024. URL https://openreview.net/forum?id=zSf8PJyQb2

    2024

  39. [48]

    Almanacs: A simulatability benchmark for language model explainability

    Mills, E., Su, S., Russell, S., and Emmons, S. Almanacs: A simulatability benchmark for language model explainability. CoRR, abs/2312.12747, 2023

    2023 arXiv

  40. [49]

    Missed causes and ambiguous effects: Counterfactuals pose challenges for interpreting neural networks

    Mueller, A. Missed causes and ambiguous effects: Counterfactuals pose challenges for interpreting neural networks. In ICML 2024 Workshop on Mechanistic Interpretability, 2024. URL https://openreview.net/forum?id=pJs3ZiKBM5

    2024

  41. [50]

    S., Sun, J., Todd, E., Bau, D., and Belinkov, Y

    Mueller, A., Brinkmann, J., Li, M., Marks, S., Pal, K., Prakash, N., Rager, C., Sankaranarayanan, A., Sharma, A. S., Sun, J., Todd, E., Bau, D., and Belinkov, Y. The quest for the right mediator: A history, survey, and theoretical grounding of causal interpretability. CoRR, ab...

    2024

  42. [51]

    Attribution Patching : Activation Patching At Industrial Scale , 2023

    Nanda, N. Attribution Patching : Activation Patching At Industrial Scale , 2023. URL https://www.neelnanda.io/mechanistic-interpretability/attribution-patching

    2023

  43. [52]

    Progress measures for grokking via mechanistic interpretability

    Nanda, N., Chan, L., Lieberum, T., Smith, J., and Steinhardt, J. Progress measures for grokking via mechanistic interpretability. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=9XFSbDPmdW

    2023

  44. [53]

    Arithmetic without algorithms: Language models solve math with a bag of heuristics

    Nikankin, Y., Reusch, A., Mueller, A., and Belinkov, Y. Arithmetic without algorithms: Language models solve math with a bag of heuristics. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=O9YTt26r2P

    2025

  45. [54]

    Norlund, T., Hagstr \"o m, L., and Johansson, R. Transferring knowledge from vision to language: How to achieve it and how to measure it? In Bastings, J., Belinkov, Y., Dupoux, E., Giulianelli, M., Hupkes, D., Pinter, Y., and Sajjad, H. (eds.), Proceedings of the Fourth Blackb...

    2021 doi

  46. [55]

    Zoom in: An introduction to circuits

    Olah, C., Cammarata, N., Schubert, L., Goh, G., Petrov, M., and Carter, S. Zoom in: An introduction to circuits. Distill, 2020. doi:10.23915/distill.00024.001. https://distill.pub/2020/circuits/zoom-in

    2020 doi

  47. [56]

    Chat GPT

    OpenAI. Chat GPT . https://openai.com/chatgpt, 2022

    2022

  48. [57]

    T he W orld of an O ctopus: H ow R eporting B ias I nfluences a L anguage M odel`s P erception of C olor

    Paik, C., Aroca-Ouellette, S., Roncone, A., and Kann, K. T he W orld of an O ctopus: H ow R eporting B ias I nfluences a L anguage M odel`s P erception of C olor. In Moens, M.-F., Huang, X., Specia, L., and Yih, S. W.-t. (eds.), Proceedings of the 2021 Conference on Empirical ...

    2021 doi

  49. [58]

    Direct and indirect effects

    Pearl, J. Direct and indirect effects. In Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence, UAI'01, pp.\ 411–420, San Francisco, CA, USA, 2001. Morgan Kaufmann Publishers Inc. ISBN 1558608001

    2001

  50. [59]

    R., Haklay, T., Belinkov, Y., and Bau, D

    Prakash, N., Shaham, T. R., Haklay, T., Belinkov, Y., and Bau, D. Fine-tuning enhances existing mechanisms: A case study on entity tracking. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net, 2024. ...

    2024

  51. [60]

    Language models are unsupervised multitask learners

    Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. Language models are unsupervised multitask learners. Blog post, 2019. URL https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf

    2019

  52. [61]

    Toward transparent AI: A survey on interpreting the inner structures of deep neural networks

    R \" a uker, T., Ho, A., Casper, S., and Hadfield - Menell, D. Toward transparent AI: A survey on interpreting the inner structures of deep neural networks. In 2023 IEEE Conference on Secure and Trustworthy Machine Learning, SaTML 2023, Raleigh, NC, USA, February 8-10, 2023 , ...

    2023

  53. [62]

    G., Hardin, C., Bhupatiraju, S., Hussenot, L., Mesnard, T., Shahriari, B., Ram \'e , A., et al

    Riviere, M., Pathak, S., Sessa, P. G., Hardin, C., Bhupatiraju, S., Hussenot, L., Mesnard, T., Shahriari, B., Ram \'e , A., et al. Gemma 2: Improving open language models at a practical size. arXiv:2408.00118, 2024. URL https://arxiv.org/abs/2408.00118

    2024 arXiv

  54. [63]

    and Wiegreffe, S

    Saphra, N. and Wiegreffe, S. Mechanistic? In The 7th BlackboxNLP Workshop, 2024. URL https://openreview.net/forum?id=schAf4BPtD

    2024

  55. [64]

    R., Materzynska, J., Chowdhury, N., Li, S., Andreas, J., Bau, D., and Torralba, A

    Schwettmann, S., Shaham, T. R., Materzynska, J., Chowdhury, N., Li, S., Andreas, J., Bau, D., and Torralba, A. FIND: A function description benchmark for evaluating interpretability methods. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), A...

    2023

  56. [65]

    J., Casper, S., Tegmark, M., Saunders, W., Bau, D., Todd, E., Geiger, A., Geva, M., Hoogland, J., Murfet, D., and McGrath, T

    Sharkey, L., Chughtai, B., Batson, J., Lindsey, J., Wu, J., Bushnaq, L., Goldowsky-Dill, N., Heimersheim, S., Ortega, A., Bloom, J., Biderman, S., Garriga-Alonso, A., Conmy, A., Nanda, N., Rumbelow, J., Wattenberg, M., Schoots, N., Miller, J., Michaud, E. J., Casper, S., Tegma...

    2025 arXiv

  57. [66]

    Hypothesis testing the circuit hypothesis in LLM s

    Shi, C., Beltran-Velez, N., Nazaret, A., Zheng, C., Garriga-Alonso, A., Jesson, A., Makar, M., and Blei, D. Hypothesis testing the circuit hypothesis in LLM s. In ICML 2024 Workshop on Mechanistic Interpretability, 2024. URL https://openreview.net/forum?id=ibSNv9cldu

    2024

  58. [67]

    Neural and conceptual interpretation of PDP models

    Smolensky, P. Neural and conceptual interpretation of PDP models. In McClelland, J. L., Rumelhart, D. E., and the PDP Research Group (eds.), Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Psychological and Biological Models, volume 2, pp.\ 39...

    1986

  59. [68]

    A mechanistic interpretation of arithmetic reasoning in language models using causal mediation analysis

    Stolfo, A., Belinkov, Y., and Sachan, M. A mechanistic interpretation of arithmetic reasoning in language models using causal mediation analysis. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp.\ 7035--7052, 2023

    2023

  60. [69]

    Axiomatic attribution for deep networks

    Sundararajan, M., Taly, A., and Yan, Q. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML'17, pp.\ 3319–3328. JMLR.org, 2017

    2017

  61. [70]

    Attribution patching outperforms automated circuit discovery

    Syed, A., Rager, C., and Conmy, A. Attribution patching outperforms automated circuit discovery. In Belinkov, Y., Kim, N., Jumelet, J., Mohebbi, H., Mueller, A., and Chen, H. (eds.), Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NL...

    2024 doi

  62. [71]

    J., Geiger, A., and Nanda, N

    Tigges, C., Hollinsworth, O. J., Geiger, A., and Nanda, N. Linear representations of sentiment in large language models. CoRR, abs/2310.15154, 2023. URL https://arxiv.org/abs/2310.15154

    2023 arXiv

  63. [72]

    Investigating gender bias in language models using causal mediation analysis

    Vig, J., Gehrmann, S., Belinkov, Y., Qian, S., Nevo, D., Singer, Y., and Shieber, S. Investigating gender bias in language models using causal mediation analysis. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural Information Proces...

    2020

  64. [73]

    R., Variengien, A., Conmy, A., Shlegeris, B., and Steinhardt, J

    Wang, K. R., Variengien, A., Conmy, A., Shlegeris, B., and Steinhardt, J. Interpretability in the wild: a circuit for indirect object identification in GPT-2 small. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 ....

    2023

  65. [74]

    Answer, assemble, ace: Understanding how LM s answer multiple choice questions

    Wiegreffe, S., Tafjord, O., Belinkov, Y., Hajishirzi, H., and Sabharwal, A. Answer, assemble, ace: Understanding how LM s answer multiple choice questions. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=6NNA0MxhCH

    2025

  66. [75]

    Wu, Z., Geiger, A., Icard, T., Potts, C., and Goodman, N. D. Interpretability at scale: Identifying causal mechanisms in alpaca. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems 36: Annual Co...

    2023

  67. [76]

    D., and Potts, C

    Wu, Z., Geiger, A., Arora, A., Huang, J., Wang, Z., Goodman, N., Manning, C. D., and Potts, C. pyvene: A library for understanding and improving pytorch models via interventions. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computa...

    2024

  68. [77]

    D., and Potts, C

    Wu, Z., Arora, A., Geiger, A., Wang, Z., Huang, J., Jurafsky, D., Manning, C. D., and Potts, C. Ax B ench: Steering LLMs ? even simple baselines outperform sparse autoencoders. CoRR, abs/2501.17148, 2025. URL https://arxiv.org/abs/2501.17148

    2025 arXiv

  69. [78]

    Qwen2.5 technical report

    Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., et al. Qwen2.5 technical report. arXiv:2412.15115, 2024. URL https://arxiv.org/abs/2412.15115

    2024 arXiv

  70. [79]

    and Nanda, N

    Zhang, F. and Nanda, N. Towards best practices of activation patching in language models: Metrics and methods. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=Hf17y6u9BC

    2024

  71. [80]

    Interpreting and improving large language models in arithmetic calculation

    Zhang, W., Wan, C., Zhang, Y., ming Cheung, Y., Tian, X., Shen, X., and Ye, J. Interpreting and improving large language models in arithmetic calculation. In Forty-first International Conference on Machine Learning, 2024. URL https://openreview.net/forum?id=CfOtiepP8s

    2024

  72. [81]

    MQ u AKE : Assessing knowledge editing in language models via multi-hop questions

    Zhong, Z., Wu, Z., Manning, C., Potts, C., and Chen, D. MQ u AKE : Assessing knowledge editing in language models via multi-hop questions. In Bouamor, H., Pino, J., and Bali, K. (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp....

    2023 doi

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.