Pith. sign in

REVIEW 4 major objections 5 minor 29 references

GPT-2 small solves opposite syllogisms by suppressing the prompt's truth token and using that suppression to boost the opposite token.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

GPT-2 small performs syllogisms through truth-copying attention heads and a suppression-plus-MLP pathway that can output a negated truth value.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Plausible negation story in GPT-2 small, but the faithfulness numbers are undefined and the path-patching corruption is left unspecified; worth refereeing, but major revision needed. the 4 major comments →

arxiv 2508.16109 v1 pith:FSM7TBAX submitted 2025-08-22 cs.CL cs.LG

From Indirect Object Identification to Syllogisms: Exploring Binary Mechanisms in Transformer Circuits

Karim Saraipour , Shichang Zhang This is my paper
classification cs.CL cs.LG
keywords mechanistic interpretabilitycircuit discoverysyllogistic reasoninglogical negationbinary truth valuesGPT-2path patchingattention heads
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that GPT-2 small does not merely copy or retrieve truth values; it can actually negate them, and does so by a specific two-step circuit. On 'opposite syllogism' prompts, where the correct answer never appears in the input, the model's Negative Truth Heads attend to the stated truth value and suppress its logit, while associated MLPs rescale the residual stream to push the opposite token to the top. The authors identify the same style of circuit for simple syllogisms, where three Truth Heads alone recover about 90% of full-model performance, and for complex syllogisms, where redundant inhibition and reinforcement heads implement the same binary operation through multiple pathways. Why this matters: it extends mechanistic interpretability from copying tasks like Indirect Object Identification to logical reasoning, and it gives a concrete, testable account of how a transformer can output a token it never saw in the prompt. A five-head, four-MLP circuit reaches about 85% faithfulness on the opposite task.

Core claim

GPT-2 small performs negation in the Opposite Syllogism task through an attention–MLP pathway: Negative Truth Heads (7.3, 8.10, 9.7, 10.7, 11.10) attend to the prompt's truth token, their OV circuits push that token to the bottom logits, and paired Truth Logit Rescaler MLPs shift the residual stream so the opposite token rises. Suppressing the wrong token is thus the signal for the correct, unseen token. In Simple Syllogisms, Truth Heads (7.2, 9.1, 9.9, 10.1, 10.4) copy the stated value, a three-head circuit reaching about 90% faithfulness. In Complex Syllogisms, inhibition and reinforcement heads operate on the incorrect token redundantly with the Negative Truth Heads (about 85% faithfulnes

What carries the argument

The load-bearing object is the attention–MLP negation pathway: a Negative Truth Head's QK circuit locates the prompt's truth token, its OV projection suppresses that token's logit, and the paired Truth Logit Rescaler MLP rescales the residual stream so the opposite token's logit rises. Path patching on logit difference isolates each component; the faithfulness metric (absolute difference between full-model and circuit average logit difference) measures how much of the behavior is preserved.

Load-bearing premise

The whole identification of Truth Heads, Negative Truth Heads, and their MLP partners rests on path patching—swapping a component's activations with those from a corrupted prompt and treating the drop in the model's preference for the right answer as that component's causal contribution—and the paper does not separately validate that this corruption-to-drop relationship cleanly isolates causal roles.

What would settle it

Mean-ablating head 10.7 on Opposite Syllogism prompts should eliminate the true→false flips while leaving false→true cases largely intact; if both directions change equally, then suppression of the prompt token is not the specific signal driving negation.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A three-head circuit (7.2, 9.1, 9.9, 10.1, 10.4) recovers roughly 90% of GPT-2 small's behavior on simple syllogisms, showing the task needs almost no MLP support.
  • On opposite syllogisms, the five Negative Truth Heads plus their MLPs recover about 85% of full-model performance, so the negation pathway is nearly sufficient by itself.
  • Heads that hurt the simple task—like 10.7—become causally important on the opposite task, so component importance is task-relative, not fixed.
  • The same circuits transfer to other binary pairs (good/bad, right/wrong, positive/negative, correct/incorrect), often matching or beating the full model, so the mechanism is not tied to the literal tokens 'true' and 'false'.
  • Negation is asymmetric: true→false is more reliable than false→true, and the truth-token embeddings show 'false' has larger projections, suggesting a learned prior toward false.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If suppression-as-negation is a general mechanism, then any binary antonym pair in the vocabulary could be flipped by steering a copy-suppression head's OV direction; a direct test is whether mean-ablating head 10.7 on the opposite syllogism specifically hurts false predictions more than true ones.
  • The paper's causal accounting leans on one choice of corrupted prompt distribution; testing alternative corruptions (for example, shuffling token order rather than replacing the truth value) would show whether the identified circuit roles are robust.
  • The true→false asymmetry hints at a corpus-level frequency prior; measuring logit differences on balanced true/false prompts across GPT-2 checkpoints could reveal when the bias emerges during training.
  • Larger models show weaker syllogism performance and muted negative-head effects, so transplanting the five-head negation circuit into those models via activation patching would test whether the pathway is architecture-specific or transferable.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents a mechanistic interpretability analysis of GPT-2 small on three syllogism tasks: Simple Syllogism (SS), Opposite Syllogism (OS), and Complex Syllogism (CS). The authors use path patching and logit lens to identify circuits they call Truth Heads, Negative Truth Heads, Truth Logit Rescaler MLPs, and Truth Modulation Heads. The central novel claim is that in OS, attention heads suppress the truth token present in the prompt while MLPs rescale the residual stream to promote the opposite truth token, enabling the model to output a token not present in the prompt. The authors report faithfulness percentages for the identified circuits, claim transfer to other binary token pairs (right/wrong, good/bad, positive/negative, correct/incorrect), and provide exploratory results on larger models.

Significance. If the central claims hold, this work would extend circuit analysis beyond IOI and provide evidence that copy-suppression heads can act as binary negators, with a concrete role for MLPs in logical negation. The cross-model and cross-token-pair transfer experiments are useful and give the paper a broader scope than many single-task circuit analyses. However, the current evidence is not sufficient to support the headline quantitative claims: the faithfulness metric is undefined as reported, the path-patching corruption distribution is not specified, and the causal mechanism for negation is only correlational. These issues are load-bearing because the paper's main contributions are the identified circuits and the negation mechanism.

major comments (4)
  1. [Section 3, Faithfulness definition; Abstract; Section 3.2] The faithfulness metric is defined as |ALD(M) − ALD(C)| with 'lower value indicates faithful recovery', yet the paper repeatedly reports percentages ('over 90%', 'roughly 85%'). In Section 3.1, the only concrete numbers are CSS ALD 1.9286 vs. full-model 1.8575. The defined metric gives |1.8575 − 1.9286| = 0.0711, not 90%; a ratio interpretation gives 103.8%, which is meaningless for a fidelity measure. The percentages are therefore undefined. This undermines the sufficiency claims in the abstract, introduction, and Section 3.2. Please specify the exact normalization or replace the metric with one that yields interpretable percentages.
  2. [Section 2, Path Patching; Sections 3.1–3.3] The corrupted distribution pnew is never specified for SS, OS, or CS. The paper only says it is 'designed to break task-relevant behavior'. The identification of Negative Truth Heads and the claimed SS/OS sign reversal of head 10.7 depend entirely on what corruption is used. A natural corruption that flips the premise truth value keeps the same token pair in distribution, so a head that merely copies the attended truth token will appear important in both tasks; a letter-permutation corruption would destroy the logical relation and change the patching results. Without pnew, the causal attribution is not reproducible and the central mechanism cannot be assessed. Please provide the exact corruption scheme, code/data, and ideally a synthetic-circuit or random-baseline validation.
  3. [Section 3.2, 'Mechanistic Interpretation' and Table 3] The suppression-then-promotion pathway is supported only by QK attention patterns and OV/logit-lens projections (Table 3). Path patching shows that certain MLPs are important, but there is no direct causal intervention demonstrating that the MLP output specifically increases the logit of the opposite token. The claim that 'Truth Logit Rescaler MLPs' promote the negation is therefore correlational. A targeted experiment—for example, patching the MLP output with a value that cancels the suppression, or applying logit lens immediately after the MLP with and without the attention head active—would provide the needed causal evidence.
  4. [Section 3.1–3.2 and Appendix B] The circuit selection and the faithfulness evaluation use the same logit-difference metric on the same tasks: heads are chosen by path-patching importance in LD, and faithfulness is then measured by |ΔLD|. This selection-evaluation overlap inflates apparent sufficiency. The transfer experiments in Appendix B partially address this, but the results are mixed: for Positive/Negative, the COS circuit gives ALD 1.0113 vs. full model 3.0032, a large discrepancy that is not consistent with 'often match or even outperform'. Please add held-out evaluation (different binary pairs, accuracy, or KL divergence) and compare against random circuits or ablations.
minor comments (5)
  1. [Abstract vs. Introduction] The abstract states that 'a circuit comprising five attention heads achieves over 90% of the original model's performance', while the introduction says three heads reach 90% on SS and five heads plus four MLPs reach about 85% on OS. Please align the claims to avoid contradiction.
  2. [Table 2] The caption says 'Top 3 highest-scoring query–key token pairs' but does not define how the scores are computed or averaged across prompts. Please clarify the exact quantity reported.
  3. [Figures 2 and 3] The colorbar indicates 'Logit diff. variation' as percentages, but the computation (normalization by baseline? standard deviation?) is not explained in the text or captions. Please add captions describing how the percentages are derived.
  4. [Section 3.3 and Figure 4] The text refers to 'logit contribution along the true direction' while the figure labels say 'Projection along the truth embedding' and 'Dot w Truth Embed'. Please use consistent terminology and clarify whether the projection is onto the unembedding vector or the embedding vector.
  5. [Appendix C] The appendix states that 'in all subsequent experiments analyzing MLP effects, we report results with attention paths patched in', but the main text Figures 2a and 3a do not note this. Please add a note in the main text so readers know the MLP effects are measured with attention context restored.

Circularity Check

1 steps flagged

Circuit components are selected by path-patching logit-difference effects and then evaluated with a faithfulness metric using the same logit difference on the same prompts, making the headline faithfulness percentages partly in-sample; the negation claim retains external support from transfer and cross-model experiments.

specific steps
  1. fitted input called prediction [Section 3.1 'Simple Syllogism' (Truth Heads and Circuit Faithfulness) and Section 3.2 'Opposite Syllogism' (Circuit Faithfulness)]
    "In contrast, Figure 2b reveals that several attention heads in the later layers, particularly heads 7.2, 9.1, 9.9, 10.1, and 10.4, contribute substantially to logit differences. ... This consistent behavior leads us to call these attention heads Truth Heads. ... To verify their sufficiency we build a minimal circuit CSS consisting only of the Truth Heads. CSS faithfully recovers the predictions of the model, achieving an average logit difference of 1.9286, effectively matching the performance of the GPT-2 small on the task."

    The heads and MLPs placed in CSS and COS were selected by path patching for the magnitude of their direct effect on logit difference (LD) on the same SS/OS prompt distributions. The faithfulness metric is then defined as |ALD(M) - ALD(C)| using that same LD. Consequently, evaluating the circuit on the same metric used to select its components is a partially in-sample test: the chosen components are by design the ones that most move LD, so a circuit built from them would be expected to reproduce a large share of the model's LD even before any mechanistic interpretation. The reported percentages ('90%', '85%') are not computable from the stated metric, which is defined only as an absolute difference, so the quantitative faithfulness claims cannot be independently checked. The transfer experi

full rationale

The paper does not rely on self-citation: all cited interpretability results (IOI, copy suppression) come from external authors, and no uniqueness claim is imported from the present authors' prior work. The central 'negation mechanism' is an interpretation of a known copy-suppression head applied to a binary task, with supporting cross-task and cross-model evidence. The main circularity concern is that the circuit is selected by path-patching effects on logit difference and then evaluated as 'faithful' using the same logit-difference metric on the same prompt distribution; this is a real selection-bias component, and the undefined faithfulness percentages prevent quantitative verification. However, because the circuits are transferred to other binary token pairs and to other models, the core claim is not equivalent to its inputs by construction. Score 3 reflects one partial in-sample evaluation issue rather than a forced or definitionally circular derivation.

Axiom & Free-Parameter Ledger

1 free parameters · 5 axioms · 4 invented entities

The paper's added value is empirical circuit identification. Its assumptions are interpretability methodology (path patching, logit lens, faithfulness) and hand-selected head groupings. It does not introduce physically new entities; the named components are internal interpretations with no independent evidential handle outside the paper.

free parameters (1)
  • head selection threshold
    No explicit threshold or significance criterion is given for which heads enter the circuits; the composition is chosen from path patching effect sizes, so it is a hand-selected modeling choice.
axioms (5)
  • domain assumption Path patching isolates causal effects of components.
    Used throughout Section 3; assumes corrupted-distribution patching does not create spurious effects and that the chosen corrupted distribution breaks only task-relevant behavior.
  • domain assumption Logit lens reveals component function.
    Used in Sections 3.1 and 3.2 to interpret head and MLP outputs from LayerNorm-plus-unembedding projections.
  • domain assumption The faithfulness metric is a valid measure of circuit fidelity.
    Defined in Section 3 as |ALD(M)-ALD(C)|; the paper uses it to claim sufficiency, but lower is better and no percentage conversion is supplied.
  • domain assumption Templates engage genuine syllogistic reasoning.
    Appendix A defines the templates; the paper assumes these prompts trigger logical computation rather than surface n-gram pattern matching.
  • domain assumption High QK attention pairs indicate logical matching.
    Section 3.1, Table 2 interprets high-scoring QK pairs such as (S, matches) and (is, true) as direct evidence of computing matches(S,E).
invented entities (4)
  • Truth Heads no independent evidence
    purpose: Identify and copy the correct truth value in Simple Syllogisms.
    Interpretive label for heads 7.2, 9.1, 9.9, 10.1, 10.4 based on in-paper patch and attention analysis; no outside falsifiable handle.
  • Negative Truth Heads no independent evidence
    purpose: Suppress the attended truth value to enable negation.
    Heads 7.3, 8.10, 9.7, 10.7, 11.10 described from OV top and bottom logits; behavior inferred within the paper.
  • Truth Logit Rescaler MLPs no independent evidence
    purpose: Rescale the residual stream to promote the opposite truth token.
    Claimed to complete the negation pathway, but only logit-lens before and after evidence is provided.
  • Truth Modulation Heads no independent evidence
    purpose: Reinforce or inhibit the incorrect truth value in Complex Syllogisms.
    Subdivided into Correct Truth Inhibition and Reinforcement heads based on correlation between attention and logit projections; in-paper interpretation only.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of From Indirect Object Identification to Syllogisms: Exploring Binary Mechanisms in Transformer Circuits." pith.science (2026). https://pith.science/paper/FSM7TBAX

@misc{pith2026250816109,
  author       = {Pith},
  title        = {Pith review of: From Indirect Object Identification to Syllogisms: Exploring Binary Mechanisms in Transformer Circuits},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FSM7TBAX}},
  note         = {Machine review of arXiv:2508.16109}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Transformer-based language models (LMs) can perform a wide range of tasks, and mechanistic interpretability (MI) aims to reverse engineer the components responsible for task completion to understand their behavior. Previous MI research has focused on linguistic tasks such as Indirect Object Identification (IOI). In this paper, we investigate the ability of GPT-2 small to handle binary truth values by analyzing its behavior with syllogistic prompts, e.g., "Statement A is true. Statement B matches statement A. Statement B is", which requires more complex logical reasoning compared to IOI. Through our analysis of several syllogism tasks of varying difficulty, we identify multiple circuits that mechanistically explain GPT-2's logical-reasoning capabilities and uncover binary mechanisms that facilitate task completion, including the ability to produce a negated token not present in the input prompt through negative heads. Our evaluation using a faithfulness metric shows that a circuit comprising five attention heads achieves over 90% of the original model's performance. By relating our findings to IOI analysis, we provide new insights into the roles of specific attention heads and MLPs in LMs. These insights contribute to a broader understanding of model reasoning and support future research in mechanistic interpretability.

Figures

Figures reproduced from arXiv: 2508.16109 by Karim Saraipour, Shichang Zhang.

Figure 1
Figure 1. Figure 1: Opposite Syllogism Circuit. The blue arrow represents queries, and the yellow arrow represents values. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Path Patching and QK Analysis on Simple Syllogism Prompts. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Path Patching and QK Analysis on Opposite Syllogism Prompts. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: The Truth Modulation group with refined truth types [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Binary task results of Right/Wrong 0 5 10 10 8 6 4 2 0 −20% −10% 0% 10% 20% Logit diff. variation Layer (a) Simple Syllogism with Correct/Incorrect 0 5 10 10 8 6 4 2 0 −80% −60% −40% −20% 0% 20% 40% 60% 80% Logit diff. variation Layer (b) Opposite Syllogism with Correct/Incorrect [PITH_FULL_IMAGE:figures/full_fig_p013_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Binary task results of Correct/Incorrect [PITH_FULL_IMAGE:figures/full_fig_p013_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Binary task results of Positive/Negative [PITH_FULL_IMAGE:figures/full_fig_p013_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Binary task results of Good/Bad 14 [PITH_FULL_IMAGE:figures/full_fig_p014_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Path patching MLPs in the opposite syllogism task. (a) shows effects when MLPs are patched with attention [PITH_FULL_IMAGE:figures/full_fig_p015_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Path patching MLPs in the simple syllogism task. (a) shows effects when MLPs are patched with attention [PITH_FULL_IMAGE:figures/full_fig_p015_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Direct effects of attention heads and MLPs for GPT-2 XL across syllogism tasks. [PITH_FULL_IMAGE:figures/full_fig_p016_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Direct effects of attention heads and MLPs for Pythia 1.4B across syllogism tasks. [PITH_FULL_IMAGE:figures/full_fig_p016_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Direct effects of attention heads and MLPs for LLaMA 3.2B across syllogism tasks. [PITH_FULL_IMAGE:figures/full_fig_p017_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Direct effects of attention heads and MLPs for Qwen 1.7B across syllogism tasks. [PITH_FULL_IMAGE:figures/full_fig_p017_14.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

29 extracted references · 8 canonical work pages · 4 internal anchors

  1. [1]

    Interpretability in the wild: a circuit for indirect object identification in gpt-2 small, 2022

    Kevin Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. Interpretability in the wild: a circuit for indirect object identification in gpt-2 small, 2022. URL https://arxiv.org/abs/2211.00593

    Pith/arXiv arXiv 2022

  2. [2]

    How does gpt-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model, 2023

    Michael Hanna, Ollie Liu, and Alexandre Variengien. How does gpt-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model, 2023. URL https://arxiv.org/abs/2305.00586

    Pith/arXiv arXiv 2023

  3. [3]

    Circuit component reuse across tasks in transformer language models, 2024

    Jack Merullo, Carsten Eickhoff, and Ellie Pavlick. Circuit component reuse across tasks in transformer language models, 2024. URL https://arxiv.org/abs/2310.08744

    Pith/arXiv arXiv 2024

  4. [4]

    Refusal in language models is mediated by a single direction, 2024

    Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. Refusal in language models is mediated by a single direction, 2024. URL https://arxiv.org/abs/2406.11717

    Pith/arXiv arXiv 2024

  5. [5]

    Universal neurons in gpt2 language models, 2024

    Wes Gurnee, Theo Horsley, Zifan Carl Guo, Tara Rezaei Kheirkhah, Qinyi Sun, Will Hathaway, Neel Nanda, and Dimitris Bertsimas. Universal neurons in gpt2 language models, 2024. URL https://arxiv.org/abs/2401.12181

    Pith/arXiv arXiv 2024

  6. [6]

    Interpreting gpt: The logit lens, 2020

    Nostalgebraist. Interpreting gpt: The logit lens, 2020. URL https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens

    work page 2020

  7. [7]

    A mathematical framework for transformer circuits

    Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. A...

    2021

  8. [8]

    Prior Analytics

    Aristotle. Prior Analytics. Oxford University Press, Oxford, UK, c. 350 BC

    work page

  9. [9]

    Copy suppression: Comprehensively understanding an attention head, 2023

    Callum McDougall, Arthur Conmy, Cody Rushing, Thomas McGrath, and Neel Nanda. Copy suppression: Comprehensively understanding an attention head, 2023. URL https://arxiv.org/abs/2310.04625

    Pith/arXiv arXiv 2023

  10. [10]

    The hydra effect: Emergent self-repair in language model computations, 2023

    Thomas McGrath, Matthew Rahtz, Janos Kramar, Vladimir Mikulik, and Shane Legg. The hydra effect: Emergent self-repair in language model computations, 2023. URL https://arxiv.org/abs/2307.15771

    Pith/arXiv arXiv 2023

  11. [11]

    Attribution patching: Activation patching at industrial scale, 2023

    Neel Nanda. Attribution patching: Activation patching at industrial scale, 2023. URL https://www.neelnanda.io/mechanistic-interpretability/attribution-patching

    work page 2023

  12. [12]

    Locating and editing factual associations in gpt, 2023

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt, 2023. URL https://arxiv.org/abs/2202.05262

    Pith/arXiv arXiv 2023

  13. [13]

    Direct and Indirect Effects, page 373–392

    Judea Pearl. Direct and Indirect Effects, page 373–392. Association for Computing Machinery, New York, NY, USA, 1 edition, 2022. ISBN 9781450395861. URL https://doi.org/10.1145/3501714.3501736

    arXiv 2022

  14. [14]

    Investigating gender bias in language models using causal mediation analysis

    Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. Investigating gender bias in language models using causal mediation analysis. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 12388--12401. Curran Assoc...

    work page 2020

  15. [15]

    Atp*: An efficient and scalable method for localizing llm behaviour to components, 2024

    János Kramár, Tom Lieberum, Rohin Shah, and Neel Nanda. Atp*: An efficient and scalable method for localizing llm behaviour to components, 2024. URL https://arxiv.org/abs/2403.00745

    Pith/arXiv arXiv 2024

  16. [16]

    Towards monosemanticity: Decomposing language models with dictionary learning

    Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E Burke, Tristan Hume, Shan Carter, Tom Henighan, and Ch...

    2023

  17. [17]

    Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller

    Samuel Marks, Can Rager, Eric J. Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller. Sparse feature circuits: Discovering and editing interpretable causal graphs in language models, 2025. URL https://arxiv.org/abs/2403.19647

    Pith/arXiv arXiv 2025

  18. [18]

    Neuron Shapley: Discovering the Responsible Neurons

    Amirata Ghorbani and James Zou. Neuron shapley: Discovering the responsible neurons, 2020. URL https://arxiv.org/abs/2002.09815

    work page internal anchor Pith review Pith/arXiv arXiv 2020

  19. [19]

    Steven Cao, Victor Sanh, and Alexander M. Rush. Low-complexity probing via finding subnetworks, 2021. URL https://arxiv.org/abs/2104.03514

    Pith/arXiv arXiv 2021

  20. [20]

    Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Masks

    Róbert Csordás, Sjoerd van Steenkiste, and Jürgen Schmidhuber. Are neural nets modular? inspecting functional modularity through differentiable weight masks, 2021. URL https://arxiv.org/abs/2010.02066

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  21. [21]

    Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowycz, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexander W

    Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowycz, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexander W. Kocurek, Ali Safaya, Ali Tazarv, Alice Xiang, Alicia Parrish, Allen Nie, Aman Hussain, Amand...

    Pith/arXiv arXiv 2023

  22. [22]

    Adaptive circuit behavior and generalization in mechanistic interpretability, 2024

    Jatin Nainani, Sankaran Vaidyanathan, AJ Yeung, Kartik Gupta, and David Jensen. Adaptive circuit behavior and generalization in mechanistic interpretability, 2024. URL https://arxiv.org/abs/2411.16105

    Pith/arXiv arXiv 2024

  23. [23]

    Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adrià Garriga-Alonso

    Arthur Conmy, Augustine N. Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adrià Garriga-Alonso. Towards automated circuit discovery for mechanistic interpretability, 2023. URL https://arxiv.org/abs/2304.14997

    Pith/arXiv arXiv 2023

  24. [24]

    A Systematic Comparison of Syllogistic Reasoning in Humans and Language Models

    Tiwalayo Eisape, MH Tessler, Ishita Dasgupta, Fei Sha, Sjoerd van Steenkiste, and Tal Linzen. A systematic comparison of syllogistic reasoning in humans and language models, 2024. URL https://arxiv.org/abs/2311.00445

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  25. [25]

    Evaluating Large Language Models with NeuBAROCO: Syllogistic Reasoning Ability and Human-like Biases

    Risako Ando, Takanobu Morishita, Hirohiko Abe, Koji Mineshima, and Mitsuhiro Okada. Evaluating large language models with neubaroco: Syllogistic reasoning ability and human-like biases, 2023. URL https://arxiv.org/abs/2306.12567

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  26. [26]

    A mechanistic interpretation of syllogistic reasoning in auto-regressive language models, 2025

    Geonhee Kim, Marco Valentino, and André Freitas. A mechanistic interpretation of syllogistic reasoning in auto-regressive language models, 2025. URL https://arxiv.org/abs/2408.08590

    Pith/arXiv arXiv 2025

  27. [27]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

  28. [28]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

  29. [29]

    From an IOI perspective, the Negative Truth Heads were initially interpreted as negative copy heads due to their tendency to replicate the tokens they attend to

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...

    arXiv

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.